Geometric Structure Of Data Research Articles

Graph Convolutional Neural Networks with Geometric and Discrimination information

In recent years, geometric deep learning methods have been proposed, which are called Graph Convolutional Neural Networks (GCNNs). GCNNs not only can extract effective features like the classical CNN, but also can effectively reflect the true geometric structure of original data. Although GCNNs consider the geometric structure of original data, they construct the same feature graph to perform graph convolution, and ignore the difference between the local structures of different samples. Therefore, a novel Graph Convolutional Neural Network with Geometric and Discrimination information (GDGCNN) is proposed, which integrates traditional machine learning ideas to further improve the performance of feature extraction. In order to exploit differences between the local structures of different samples and make full use of the geometric structure of original data, GDGCNN constructs different feature graphs for different training batches to fully exploit the local geometry of data. Moreover, the discriminant regularization is introduced into GDGCNN to effectively utilize the discriminant information contained in original data. Therefore, GDGCNN has good discriminative ability and robustness. The experimental results show that GDGCNN can perform feature extraction tasks very well, and it is superior to some existing methods for classification in terms of accuracy and F1-Score.

Brain Tumor MR Image Classification Using Convolutional Dictionary Learning With Local Constraint.

Brain tumor image classification is an important part of medical image processing. It assists doctors to make accurate diagnosis and treatment plans. Magnetic resonance (MR) imaging is one of the main imaging tools to study brain tissue. In this article, we propose a brain tumor MR image classification method using convolutional dictionary learning with local constraint (CDLLC). Our method integrates the multi-layer dictionary learning into a convolutional neural network (CNN) structure to explore the discriminative information. Encoding a vector on a dictionary can be considered as multiple projections into new spaces, and the obtained coding vector is sparse. Meanwhile, in order to preserve the geometric structure of data and utilize the supervised information, we construct the local constraint of atoms through a supervised k-nearest neighbor graph, so that the discrimination of the obtained dictionary is strong. To solve the proposed problem, an efficient iterative optimization scheme is designed. In the experiment, two clinically relevant multi-class classification tasks on the Cheng and REMBRANDT datasets are designed. The evaluation results demonstrate that our method is effective for brain tumor MR image classification, and it could outperform other comparisons.

I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting

3D object classification has attracted appealing attentions in academic researches and industrial applications. However, most existing methods need to access the training data of past 3D object classes when facing the common real-world scenario: new classes of 3D objects arrive in a sequence. Moreover, the performance of advanced approaches degrades dramatically for past learned classes (i.e., catastrophic forgetting), due to the irregular and redundant geometric structures of 3D point cloud data. To address these challenges, we propose a new Incremental 3D Object Learning (i.e., I3DOL) model, which is the first exploration to learn new classes of 3D object continually. Specifically, an adaptive-geometric centroid module is designed to construct discriminative local geometric structures, which can better characterize the irregular point cloud representation for 3D object. Afterwards, to prevent the catastrophic forgetting brought by redundant geometric information, a geometric-aware attention mechanism is developed to quantify the contributions of local geometric structures, and explore unique 3D geometric characteristics with high contributions for classes incremental learning. Meanwhile, a score fairness compensation strategy is proposed to further alleviate the catastrophic forgetting caused by unbalanced data between past and new classes of 3D object, by compensating biased prediction for new classes in the validation phase. Experiments on 3D representative datasets validate the superiority of our I3DOL framework.

Change detection in remote sensing images based on manifold regularized joint non-negative matrix factorization

A novel and effective change detection method based on manifold regularized joint non-negative matrix factorization (MJNMF) framework is proposed in this paper, which detects the changes that occurred in multi-temporal remote sensing images. Most change detection methods, including dictionary learning, principal component analysis (PCA), etc., do not consider the non-negativity among image pixels. However, image itself is a non-negative signal, and the non-negative constraint has better interpretability in practical applications. Nonnegative Matrix Factorization, which incorporates the non-negativity constraint and thus learns object parts, obtains the parts-based representation as well as enhancing the interpretability of the issue correspondingly. In this paper, our proposed approach based on MJNMF framework aims to establish a pair of joint basis matrices by unchanged training samples from unchanged area. Then, unchanged pixels can be well reconstructed by the corresponding basis matrix, while changed pixels cannot be reconstructed from the basis matrix corresponding to the knowledge of unchanged samples, or a larger reconstruction error can be generated even if changed pixels are reconfigurable. In order to suppress similar information and highlight different information, the cross-reconstruction error is used to generate the difference image. Finally, the binary image is obtained by the robust fuzzy local information c-means (FLICM) clustering algorithm. In addition, inspired by manifold learning, we incorporate manifold regularization into the proposed method to keep the geometric structure of data and improve the accuracy of change detection. Experimental results obtained on simulated and real remote sensing images confirm the effectiveness of the proposed method.

Weighted manifold regularized sparse representation of featured injected details for pansharpening

ABSTRACT Sparse representation (SR)-based pansharpening methods which combine the dictionary and estimated sparse coefficients have achieved visually and quantitatively great results in pansharpening problem these days. And the details injection (ID)-based methods can receive comparable images by sharpening the multispectral bands through adding the proper spatial details from panchromatic (PAN) images. Recently, method based on sparse representation of injected details (SR-D) which combines the SR and ID points out a new way forward for pansharpening. In this direction, manifold regularized sparse representation of injected details (MR-SR-D) which introducing a manifold regularization (MR) into the former SR-D model have improved the quality of pansharpened images greatly utilizing a graph Laplacian to incorporate the locally geometrical structure of the multispectral data. However, due to the lack of spatial information in PAN, and the use of higher-order features similarity with the original multispectral images, the resulting images still have spatial and spectral distortion. Thus, in this paper, we propose a new method to enhance the spatial resolution of aiming image by adding the weighted local geometrical structure of PAN and multispectral images, and improve the spectral resolution by joining the higher-order structure connection between multispectral and aiming images to the MR-SR-D method which can be called as weighted manifold regularized (WMR) sparse representation of featured injected details method (WMR-SR-FD). Experimental results using IKONOS, QuickBird and WorldView2 data show that the proposed method can achieve remarkable spectral and spatial quality.

Self-supervised graph convolutional clustering by preserving latent distribution

Node clustering has attracted widespread concern due to the fact that graph structure helps to seek the geometric structure of data. Existing graph convolutional networks (GCNs) based node clustering methods usually contain two steps. First, learning latent nodes representations. Second, inferring the unknown labels. Despite the promising preliminary results, it cannot take full advantage of the information embedded in current pseudo clustering labels, resulting in suboptimal performance. In fact, it is the truth we do not know the true label of the node in clustering, but the pseudo clustering label of the node generated by each iteration is known. Under this condition, although the pseudo labels of some nodes are incorrect, the partial correct label information is useful and significant. Moreover, GCNs based node clustering aims to learn satisfactory low dimensional node representations which are usually used for clustering. However, existing GCNs based clustering methods fail to take into account distribution consistency between the raw data space and the latent space of nodes representations, resulting in insufficient representations. To address the above problems, we propose a novel GCN-based node clustering method. By introducing a self-supervision module, it can employ the good property of pseudo clustering labels to self-supervise the learning of node representations. Moreover, a latent distribution preserving term, which is measured by KL divergence, is employed to help the latent representations of the same sample to further have a consistent distribution in dimension space as well as in the original dimension space. We formulate the above two concerns into a unified optimization framework. Experimental results on several public datasets indicate our method is effective compared with the state-of-the-art node clustering methods.

Inductive Geometric Matrix Midranges

Covariance data as represented by symmetric positive definite (SPD) matrices are ubiquitous throughout technical study as efficient descriptors of interdependent systems. Euclidean analysis of SPD matrices, while computationally fast, can lead to skewed and even unphysical interpretations of data. Riemannian methods preserve the geometric structure of SPD data at the cost of expensive eigenvalue computations. In this paper, we propose a geometric method for unsupervised clustering of SPD data based on the Thompson metric. This technique relies upon a novel "inductive midrange" centroid computation for SPD data, whose properties are examined and numerically confirmed. We demonstrate the incorporation of the Thompson metric and inductive midrange into X-means and K-means++ clustering algorithms.

Spectral-Spatial Constrained Nonnegative Matrix Factorization for Spectral Mixture Analysis of Hyperspectral Images

Hyperspectral spectral mixture analysis (SMA), which intends to decompose mixed pixels into a collection of endmembers weighted by their corresponding fraction abundances, has been successfully used to tackle mixed-pixel problem in hyperspectral remote sensing applications. As an approach of decomposing a high-dimensional data matrix into the multiplication of two nonnegative matrices, nonnegative matrix factorization (NMF) has shown its advantages and been widely applied to SMA. Unfortunately, most of the NMF-based unmixing methods can easily lead to an unsuitable solution, due to inadequate mining of spatial and spectral information and the influence of outliers and noise. To overcome such limitations, a spatial constraint over abundance and a spectral constraint over endmembers are imposed over NMF-based unmixing model for spectral-spatial constrained unmixing. Specifically, a spatial neighborhood preserving constraint is proposed to preserve the local geometric structure of the hyperspectral data by assuming that pixels in a spatial neighborhood generally fall into a low-dimensional manifold, while a minimum spectral distance constraint is formulated to optimize endmember spectra as compact as possible. Furthermore, to handle non-Gaussian noises or outliers, an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm based loss function is ultimately adopted for the proposed spectral-spatial constrained nonnegative matrix factorization model and a projected gradient based optimization algorithm is designed for optimization. Experimental results over both synthetic and real-world datasets demonstrate that the proposed spatial and spectral constraints can certainly improve the performance of NMF-based unmixing and outperform state-of-the-art NMF-based unmixing algorithms.

Incomplete Multi-view Clustering with Joint Partition and Graph Learning

Incomplete multi-view clustering (IMC) aims to integrate the complementary information from incomplete views to improve clustering performance. Most existing IMC methods try to fill the incomplete views or directly learn a common representation based on matrix factorization or subspace learning. The former may introduce useless even noisy information especially for data with a large missing ratio. The latter relies on the initialization and ignores the geometric structure of data. To address these issues, we propose a novel Joint Partition and Graph (JPG) learning method for IMC. Specifically, JPG jointly constructs local incomplete graph matrices, generates incomplete base partition matrices, stretches them to produce a unified partition matrix, and employs it to learn a consensus graph matrix. By this means, we transform incomplete multi-view data into a unified partition space and obtain the consensus graph in a mutual reinforcement manner. Moreover, a partition fusion strategy can allocate a large weight to the stretched base partition that is close to the unified matrix. The objective function is optimized in an alternating optimization fashion. Experimental results on several benchmark datasets demonstrate the effectiveness and superiority of JPG than the state-of-the-art baselines

Low-Rank Joint Embedding and Its Application for Robust Process Monitoring

Industrial data are in general corrupted by noises and outliers. In this context, robustness to the contaminated data is a challenging issue in process monitoring. In this article, a novel method named low-rank joint embedding is proposed for robust process monitoring. By learning a low-rank coefficient matrix, low-rank joint embedding can capture the global structure of the original data and alleviate the negative effect of outliers, making the monitoring results more reliable. Moreover, a manifold regularization is introduced to preserve the local geometric structure of data, which enables the extracted low-dimensional representation of data to be more faithful and informative to enhance the monitoring capability. Based on projection learning, the low-rank joint embedding can learn an explicit projection that transforms the data not involved in the training data into the low-dimensional space, avoiding the out-of-sample problem. Furthermore, a reconstruction-based contribution plots based on the low-rank joint embedding is developed to identify the potential faulty variables. Case studies on the Tennessee Eastman process and a real industrial application demonstrate the effectiveness of the proposed monitoring approach.

Generalized Multi-manifold Graph Ensemble Embedding for Multi-View Dimensionality Reduction

In this paper, we propose a new dimension reduction (DR) algorithm called ensemble graph-based locality preserving projections (EGLPP); to overcome the neighborhood size k sensitivity in locally preserving projections (LPP). EGLPP constructs a homogeneous ensemble of adjacency graphs by varying neighborhood size k and finally uses the integrated embedded graph to optimize the low-dimensional projections. Furthermore, to appropriately handle the intrinsic geometrical structure of the multi-view data and overcome the dimensionality curse, we propose a generalized multi-manifold graph ensemble embedding framework (MLGEE). MLGEE aims to utilize multi-manifold graphs for the adjacency estimation with automatically weight each manifold to derive the integrated heterogeneous graph. Experimental results on various computer vision databases verify the effectiveness of proposed EGLPP and MLGEE over existing comparative DR methods.

Learning Dual Geometric Low-Rank Structure for Semisupervised Hyperspectral Image Classification.

Most of the available graph-based semisupervised hyperspectral image classification methods adopt the cluster assumption to construct a Laplacian regularizer. However, they sometimes fail due to the existence of mixed pixels whose recorded spectra are a combination of several materials. In this paper, we propose a geometric low-rank Laplacian regularized semisupervised classifier, by exploring both the global spectral geometric structure and local spatial geometric structure of hyperspectral data. A new geometric regularized Laplacian low-rank representation (GLapLRR)-based graph is developed to evaluate spectral-spatial affinity of mixed pixels. By revealing the global low-rank and local spatial structure of images via GLapLRR, the constructed graph has the characteristics of spatial-spectral geometry description, robustness, and low sparsity, from which a more accurate classification of mixed pixels can be achieved. The proposed method is experimentally evaluated on three real hyperspectral datasets, and the results show that the proposed method outperforms its counterparts, when only a small number of labeled instances are available.

Robust distribution-based nonnegative matrix factorizations for dimensionality reduction

As a popular dimensionality-reduction technique, nonnegative matrix factorization (NMF) has been widely researched since it is consistent with human cognitive processes in the psychology and physiology. This paper presents a novel NMF framework, called robust distribution-based NMF (RDNMF), to learn the robustly discriminative representations for data. In this RDNMF, a Kullback–Leibler divergence to measure the similarity between the data and representations is introduced, which fully preserves the geometrical structure of data. Meanwhile, this RDNMF employs the l2,1-norm loss to reduce the influence of noise and outliers. This paper further proposes a semi-supervised RDNMF (SRDNMF) by enforcing the representations of labeled points in the same class to be aligned on the same axis. The proposed RDNMF and SRDNMF are solved by the modified multiplicative update rules. Clustering experiments on seven benchmark datasets demonstrate the effectiveness of our methods in comparison to other state-of-the-art methods.

Functional Aggregate Queries with Additive Inequalities

Motivated by fundamental applications in databases and relational machine learning, we formulate and study the problem of answering functional aggregate queries (FAQ) in which some of the input factors are defined by a collection of additive inequalities between variables. We refer to these queries as FAQ-AI for short. To answer FAQ-AI in the Boolean semiring, we define relaxed tree decompositions and relaxed submodular and fractional hypertree width parameters. We show that an extension of the InsideOut algorithm using Chazelle’s geometric data structure for solving the semigroup range search problem can answer Boolean FAQ-AI in time given by these new width parameters. This new algorithm achieves lower complexity than known solutions for FAQ-AI. It also recovers some known results in database query answering. Our second contribution is a relaxation of the set of polymatroids that gives rise to the counting version of the submodular width, denoted by #subw. This new width is sandwiched between the submodular and the fractional hypertree widths. Any FAQ and FAQ-AI over one semiring can be answered in time proportional to #subw and respectively to the relaxed version of #subw. We present three applications of our FAQ-AI framework to relational machine learning: k -means clustering, training linear support vector machines, and training models using non-polynomial loss. These optimization problems can be solved over a database asymptotically faster than computing the join of the database relations.

Multi-label feature selection with constraint regression and adaptive spectral graph

Like single-label learning, multi-label learning also suffers from the curse of dimensionality. Due to the existence of high-dimensional data, feature selection as a preprocessing tool always plays a key position in the multi-label learning. Although a variety of multi-label feature selection methods have been developed, they neglect to consider the redundancy of features, thereby degrading learning performance. To address this problem, we present a novel multi-label feature selection approach with uncorrelated regression and adaptive spectral graph. Specifically, we first construct a manifold framework with uncorrelated regression model to hunt for uncorrelated yet discriminative features, which also utilize the low-dimensional representation based on feature space to fit the label distribution. Then, a spectral graph term based on information entropy is incorporated into the manifold framework, so as to ensure the local geometric structure of data in subsequent learning process. Following the above principles, we design an objective function to achieve multi-label feature selection, and propose a detailed optimization method. Comprehensive experiments are conducted on multiple public multi-label data sets, the results show that our method is superior to other compared methods.

Trajectories, bifurcations, and pseudo-time in large clinical datasets: applications to myocardial infarction and diabetes data.

BackgroundLarge observational clinical datasets are becoming increasingly available for mining associations between various disease traits and administered therapy. These datasets can be considered as representations of the landscape of all possible disease conditions, in which a concrete disease state develops through stereotypical routes, characterized by “points of no return" and “final states" (such as lethal or recovery states). Extracting this information directly from the data remains challenging, especially in the case of synchronic (with a short-term follow-up) observations.ResultsHere we suggest a semi-supervised methodology for the analysis of large clinical datasets, characterized by mixed data types and missing values, through modeling the geometrical data structure as a bouquet of bifurcating clinical trajectories. The methodology is based on application of elastic principal graphs, which can address simultaneously the tasks of dimensionality reduction, data visualization, clustering, feature selection, and quantifying the geodesic distances (pseudo-time) in partially ordered sequences of observations. The methodology allows a patient to be positioned on a particular clinical trajectory (pathological scenario) and the degree of progression along it to be characterized with a qualitative estimate of the uncertainty of the prognosis. We developed a tool ClinTrajan for clinical trajectory analysis implemented in the Python programming language. We test the methodology in 2 large publicly available datasets: myocardial infarction complications and readmission of diabetic patients data.ConclusionsOur pseudo-time quantification-based approach makes it possible to apply the methods developed for dynamical disease phenotyping and illness trajectory analysis (diachronic data analysis) to synchronic observational data.

Joint Nonnegative Matrix Factorization Based on Sparse and Graph Laplacian Regularization for Clustering and Co-Differential Expression Genes Analysis

The explosion of multiomics data poses new challenges to existing data mining methods. Joint analysis of multiomics data can make the best of the complementary information that is provided by different types of data. Therefore, they can more accurately explore the biological mechanism of diseases. In this article, two forms of joint nonnegative matrix factorization based on the sparse and graph Laplacian regularization (SG-jNMF) method are proposed. In the method, the graph regularization constraint can preserve the local geometric structure of data.L2,1-norm regularization can enhance the sparsity among the rows and remove redundant features in the data. First, SG-jNMF1 projects multiomics data into a common subspace and applies the multiomics fusion characteristic matrix to mine the important information closely related to diseases. Second, multiomics data of the same disease are mapped into the common sample space by SG-jNMF2, and the cluster structures are detected clearly. Experimental results show that SG-jNMF can achieve significant improvement in sample clustering compared with existing joint analysis frameworks. SG-jNMF also effectively integrates multiomics data to identify co-differentially expressed genes (Co-DEGs). SG-jNMF provides an efficient integrative analysis method for mining the biological information hidden in heterogeneous multiomics data.

Efficient Data Structures for Range Shortest Unique Substring Queries

Let T[1,n] be a string of length n and T[i,j] be the substring of T starting at position i and ending at position j. A substring T[i,j] of T is a repeat if it occurs more than once in T; otherwise, it is a unique substring of T. Repeats and unique substrings are of great interest in computational biology and information retrieval. Given string T as input, the Shortest Unique Substring problem is to find a shortest substring of T that does not occur elsewhere in T. In this paper, we introduce the range variant of this problem, which we call the Range Shortest Unique Substring problem. The task is to construct a data structure over T answering the following type of online queries efficiently. Given a range [α,β], return a shortest substring T[i,j] of T with exactly one occurrence in [α,β]. We present an O(nlogn)-word data structure with O(logwn) query time, where w=Ω(logn) is the word size. Our construction is based on a non-trivial reduction allowing for us to apply a recently introduced optimal geometric data structure [Chan et al., ICALP 2018]. Additionally, we present an O(n)-word data structure with O(nlogϵn) query time, where ϵ&gt;0 is an arbitrarily small constant. The latter data structure relies heavily on another geometric data structure [Nekrich and Navarro, SWAT 2012].

Minimum cuts in geometric intersection graphs

Let D be a set of n disks in the plane. The disk graphGD for D is the undirected graph with vertex set D in which two disks are joined by an edge if and only if they intersect. The directed transmission graphGD→ for D is the directed graph with vertex set D in which there is an edge from a disk D1∈D to a disk D2∈D if and only if D1 contains the center of D2.Given D and two non-intersecting disks s,t∈D, we show that a minimum s-t vertex cut in GD or in GD→ can be found in O(n3/2polylogn) expected time. To obtain our result, we combine an algorithm for the maximum flow problem in general graphs with dynamic geometric data structures to manipulate the disks.As an application, we consider the barrier resilience problem in a rectangular domain. In this problem, we have a vertical strip S bounded by two vertical lines, Lℓ and Lr, and a collection D of disks. Let a be a point in S above all disks of D, and let b a point in S below all disks of D. The task is to find a curve from a to b that lies in S and that intersects as few disks of D as possible. Using our improved algorithm for minimum cuts in disk graphs, we can solve the barrier resilience problem in O(n3/2polylogn) expected time.

PointFusionNet: Point feature fusion network for 3D point clouds analysis

The 3D point clouds is an important type of geometric data structure, and the analysis of 3D point clouds based on deep learning is a very challenging task due to the disorder and irregularity. In existing research, RS-CNN provides an effective and promising method to obtain shape features on disordered point clouds directly, which encodes local features effectively. However, RS-CNN fails to consider point-wise features and global features, which are conducive to point clouds better. In this paper, we proposed PointFusionNet, which solves these problems effectively by fusing point-wise features, local features, and global features. We have designed Feature Fusion Convolution (FF-Conv) and Global Relationship Reasoning Module (GRRM) to build PointFusionNet. The point-wise features were fused with their corresponding local features in the FF-Conv and then mapped into a high-dimensional space to extract richer local features. The GRRM inferred the relationship between various parts, in order to capture global features for enriching the content of the feature descriptor. Therefore the PointFusionNet is suitable for point clouds classification and semantic segmentation by using the two distinctive modules. The PointFusionNet has been tested on ModelNet40 and ShapeNet part datasets, and the experiments show that PointFusionNet has a competitive advantage in shape classification and part segmentation tasks.