Global Structure Preservation Research Articles

Global Structure Preservation and Self-Representation-Based Supervised Feature Selection

Feature selection aims to select a subset of features from high-dimensional data, which can overcome the curse of dimensionality for the next dealing steps. However, the feature selection itself could face the curse of dimensionality. To overcome the above problem, in this paper, a new feature selection framework is designed according to a human processing in our daily life. In our daily life, to evaluate a candidate's ability to work, the related professional knowledge and the comprehensive ability of a candidate should be both evaluated. Actually, a candidate only with good professional knowledge often hardly solves new problems in the work. Based on the above analysis, in our new designed framework, the features are selected by evaluating its ability of global structure preservation and self-representation, which are respectively similar to the professional knowledge and comprehensive ability in evaluating candidate. As a result, the selected features can accommodate larger changes in test data. The conducted experiments validate the effectiveness of our feature selection.

Unsupervised learning: Local and global structure preservation in industrial data

This article examines the use of data mining and machine learning algorithms for knowledge discovery in industrial processes, with a focus on understanding how these methods distinguish and represent the global and local structures underlying data. The effectiveness of alternative dimensionality reduction algorithms in characterizing process operations is also explored, with an emphasis on a novel dimensionality reduction method, PaCMAP, compared to the dependable historical method of PCA and the more recent t-SNE. The article demonstrates a visualization method for identifying the variables responsible for the separation between selected clusters and studies the effects of key elements of the algorithms on cluster analysis in light of domain knowledge. Results from an industrial case study of a pyrolysis reactor validates the applicability of these methods in real-life scenarios. Overall, this article provides insights into the use of machine learning and dimensionality reduction algorithms for knowledge discovery in industrial processes.

Depth-Aware Neural Style Transfer for Videos

Temporal consistency and content preservation are the prominent challenges in artistic video style transfer. To address these challenges, we present a technique that utilizes depth data and we demonstrate this on real-world videos from the web, as well as on a standard video dataset of three-dimensional computer-generated content. Our algorithm employs an image-transformation network combined with a depth encoder network for stylizing video sequences. For improved global structure preservation and temporal stability, the depth encoder network encodes ground-truth depth information which is fused into the stylization network. To further enforce temporal coherence, we employ ConvLSTM layers in the encoder, and a loss function based on calculated depth information for the output frames is also used. We show that our approach is capable of producing stylized videos with improved temporal consistency compared to state-of-the-art methods whilst also successfully transferring the artistic style of a target painting.

MEQA: Manifold embedding quality assessment via anisotropic scaling and Kolmogorov-Smirnov test

Manifold learning methods unfold the manifold structures and embed them in a lower-dimensional space. The quality of such an embedding should be measured both qualitatively and quantitatively. The proposed manifold embedding quality assessment (MEQA) method does so by taking into account of local and global structure preservation as both are important traits of an embedding. To measure the local structure preservation MEQA uses two transformations. Initially anisotropic scaling, rotation and translation are incorporated to measure the closeness between the original and the embedded data points. In the next stage, rigid transformation is incorporated to quantify the previous transformation which involved anisotropic scaling. For quantifying the global structure preservation, the Kolmogorov-Smirnov test is applied in a distributed manner over each dimension and averaged over them. To establish the superiority of MEQA we conducted several studies over standard synthetic and real-life datasets across separate existing feature extraction techniques.

Robust sparse and low-redundancy multi-label feature selection with dynamic local and global structure preservation

Recent years, joint feature selection and multi-label learning have received extensive attention as an open problem. However, there exist three general issues in previous multi-label feature selection methods. First of all, existing methods either consider local label correlations or global label correlations when they design multi-label feature selection methods, in fact, both two types of label correlations are significant for feature selection; second, previous methods use the low-quality graph to excavate local label correlations so that the results of these methods are under-performing; third, feature redundancy is ignored by most of the sparse learning methods. To overcome these challenges, we preserve global label correlations and dynamic local label correlations by preserving graph structure. Additionally, the l2,1-norm and an inner product regularization term are imposed onto the objective function to preserve robust high row-sparsity and to select low redundant features. All the above terms are integrated into one learning framework, and then we utilize a simple yet effective scheme to optimize the framework. Experimental results demonstrate the classification superiority of the proposed method.

Multi-view multi-manifold learning with local and global structure preservation

Multi-view multi-manifold learning with local and global structure preservation

SQuadMDS: A lean Stochastic Quartet MDS improving global structure preservation in neighbor embedding like t-SNE and UMAP

Multidimensional scaling is a process that aims to embed high dimensional data into a lower-dimensional space; this process is often used for the purpose of data visualisation. Common multidimensional scaling algorithms tend to have high computational complexities, making them inapplicable on large data sets. This work introduces a stochastic, force directed approach to multidimensional scaling with a time and space complexity of O(N), with N data points. The method can be combined with force directed layouts of the family of neighbour embedding such as t-SNE, to produce embeddings that preserve both the global and the local structures of the data.Experiments assess the quality of the embeddings produced by the standalone version and its hybrid extension both quantitatively and qualitatively, showing competitive results outperforming state-of-the-art approaches. Codes are available athttps://github.com/PierreLambert3/SQuaD-MDS-and-FItSNE-hybrid.

Fuzzy Information Discrimination Measures and Their Application to Low Dimensional Embedding Construction in the UMAP Algorithm.

Dimensionality reduction techniques are often used by researchers in order to make high dimensional data easier to interpret visually, as data visualization is only possible in low dimensional spaces. Recent research in nonlinear dimensionality reduction introduced many effective algorithms, including t-distributed stochastic neighbor embedding (t-SNE), uniform manifold approximation and projection (UMAP), dimensionality reduction technique based on triplet constraints (TriMAP), and pairwise controlled manifold approximation (PaCMAP), aimed to preserve both the local and global structure of high dimensional data while reducing the dimensionality. The UMAP algorithm has found its application in bioinformatics, genetics, genomics, and has been widely used to improve the accuracy of other machine learning algorithms. In this research, we compare the performance of different fuzzy information discrimination measures used as loss functions in the UMAP algorithm while constructing low dimensional embeddings. In order to achieve this, we derive the gradients of the considered losses analytically and employ the Adam algorithm during the loss function optimization process. From the conducted experimental studies we conclude that the use of either the logarithmic fuzzy cross entropy loss without reduced repulsion or the symmetric logarithmic fuzzy cross entropy loss with sufficiently large neighbor count leads to better global structure preservation of the original multidimensional data when compared to the loss function used in the original UMAP algorithm implementation.

Deep Recursive Embedding for High-Dimensional Data.

Embedding high-dimensional data onto a low-dimensional manifold is of both theoretical and practical value. In this article, we propose to combine deep neural networks (DNN) with mathematics-guided embedding rules for high-dimensional data embedding. We introduce a generic deep embedding network (DEN) framework, which is able to learn a parametric mapping from high-dimensional space to low-dimensional space, guided by well-established objectives such as Kullback-Leibler (KL) divergence minimization. We further propose a recursive strategy, called deep recursive embedding (DRE), to make use of the latent data representations for boosted embedding performance. We exemplify the flexibility of DRE by different architectures and loss functions, and benchmarked our method against the two most popular embedding methods, namely, t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP). The proposed DRE method can map out-of-sample data and scale to extremely large datasets. Experiments on a range of public datasets demonstrated improved embedding performance in terms of local and global structure preservation, compared with other state-of-the-art embedding methods. Code is available at https://github.com/tao-aimi/DeepRecursiveEmbedding.

Parametric UMAP Embeddings for Representation and Semisupervised Learning.

UMAP is a nonparametric graph-based dimensionality reduction algorithm using applied Riemannian geometry and algebraic topology to find low-dimensional embeddings of structured data. The UMAP algorithm consists of two steps: (1) computing a graphical representation of a data set (fuzzy simplicial complex) and (2) through stochastic gradient descent, optimizing a low-dimensional embedding of the graph. Here, we extend the second step of UMAP to a parametric optimization over neural network weights, learning a parametric relationship between data and embedding. We first demonstrate that parametric UMAP performs comparably to its nonparametric counterpart while conferring the benefit of a learned parametric mapping (e.g., fast online embeddings for new data). We then explore UMAP as a regularization, constraining the latent distribution of autoencoders, parametrically varying global structure preservation, and improving classifier accuracy for semisupervised learning by capturing structure in unlabeled data.1.

Local coordinates and global structure preservation for fault detection and diagnosis

Fault detection is crucial for the safe and reliable operation of industrial processes. Manifold learning methods have been widely used by the fault detection community. However, traditional manifold learning-based fault detection methods extract either the global structure information of the given data or the local structure information hidden in the data without capturing both global and local representations, which probably deteriorates the detection capabilities of these methods. This paper proposes a new feature extraction method named local coordinates and global structure preservation (LCGSP) for fault detection, where the local tangent space information and global geometric structure of data are captured simultaneously. A distance ratio is proposed and then integrated into principal component analysis to approximate the local tangent space, and this technique can improve the accuracy of the approximated local tangent space even if the data are sparse or non-uniformly distributed. In this context, the proposed method can better preserve the local manifold structure through the use of an accurate local tangent space. Furthermore, the geodesic distances between non-neighbors are introduced to preserve the global intrinsic relationships among data points. In this way, more faithful representations of the original data can be captured, thus enhancing the detection performance of the proposed method. In addition, a Bayesian fault diagnosis strategy is developed to make decisions about fault statuses. Two case studies demonstrate the superiority of the proposed LCGSP-based fault detection and diagnosis approach.

Analyzing the quality of local and global multidimensional projections using performance evaluation planning

Among the challenges of the big data era, the analysis of high-dimensional data is still an open research area. As a result, several multidimensional projection techniques have been developed to reduce data dimensionality, becoming important visualization and visual analytics tools. In order to ensure the quality of projections, it is necessary to assess the low-dimensional embeddings by using different dataset configurations as input and analyzing evaluation metrics. However, it is not clear to the user how factors such as the number of dimensions, instances, or clusters, can affect the projection mapping and its quality regarding different projection techniques and assessment metrics. The research reported in this paper aims to clarify how much these factors affect each response variable via performance evaluation planning. We present an evaluation approach, supported by factorial design, that carries out a complete analysis, in the sense of measuring all possible combinations of all the input factors. The results of the analyses of local and global structure preservation in the projections yield a better understanding of how distinct dataset properties can influence the choice of projections based on quality metrics results.

Dual Global Structure Preservation Based Supervised Feature Selection

Dual Global Structure Preservation Based Supervised Feature Selection

Discriminative transfer learning via local and global structure preservation

The current success of supervised learning is limited on large amounts of labeled training data. Transfer learning aims to learn an adaptive classifier for the unlabeled target domain data from the labeled source domain data, which is sampled from diverse probability distributions under changing conditions. Most previous works focus on how to reduce the distribution discrepancy between two involved domains, or exploit the shared common feature by preserving the local geometric structure of samples. In this paper, we propose a modified method jointly optimizing the local and global structure preservation. The main idea is to explore common features with manifold regularization. Discriminative repulsive force model is used to improve maximum mean discrepancy, which keeps discriminative property in the local sense via labeled source domain data and alleviates the global distribution discrepancy of the different domains. Quantitative results indicate that our method performs better than other methods on 16 cross-domain experiments.

Local and Global Structure Preservation for Robust Unsupervised Spectral Feature Selection

This paper proposes a new unsupervised spectral feature selection method to preserve both the local and global structure of the features as well as the samples. Specifically, our method uses the self-expressiveness of the features to represent each feature by other features for preserving the local structure of features, and a low-rank constraint on the weight matrix to preserve the global structure among samples as well as features. Our method also proposes to learn the graph matrix measuring the similarity of samples for preserving the local structure among samples. Furthermore, we propose a new optimization algorithm to the resulting objective function, which iteratively updates the graph matrix and the intrinsic space so that collaboratively improving each of them. Experimental analysis on 12 benchmark datasets showed that the proposed method outperformed the state-of-the-art feature selection methods in terms of classification performance.

Multi-operator image retargeting with automatic integration of direct and indirect seam carving

Multi-operator image resizing can preserve important objects and structure in an image by combining multiple image resizing operators. However, traditional multi-operator methods do not take both horizontal and vertical content-aware resizing potential into consideration, which essentially leads to squeeze/stretch effect in the resultant images. In this paper, we propose a new multi-operator scheme that addresses aforementioned issue by integrating direct and indirect seam carving. Compared with previous methods, the proposed scheme remarkably reduces the cost of deciding when to change operators, by employing a newly defined image artifact measure. Furthermore, we propose a novel seam carving enhancement, named ACcumulated Energy Seam Carving (ACESC), as a basic operator to improve global structure preservation. By combining horizontal and vertical seam carving, our scheme preserves the shapes of important objects well. We present typical results to demonstrate the effectiveness of our method. User study shows that our method has high user preference.

Protein Similarity from Knot Theory: Geometric Convolution and Line Weavings

Shape similarity is one of the most elusive and intriguing questions of nature and mathematics. Proteins provide a rich domain in which to test theories of shape similarity. Proteins can match at different scales and in different arrangements. Sometimes the detection of common local structure is sufficient to infer global alignment of two proteins; at other times it provides false information. Proteins with very low sequence identity may share large substructures, or perhaps just a central core. There are even examples of proteins with nearly identical primary sequences in which alpha-helices have become beta-sheets. Shape similarity can be formulated (i) in terms of global metrics, such as RMSD or Hausdorff distance, (ii) in terms of subgraph isomorphisms, such as the detection of shared substructures with similar relative locations, or (iii) purely topologically, in terms of structure preserving transformations. Existing protein structure detection programs are built on the first two types of similarity. The third forms the foundations of knot theory. The thesis of this paper is this: Protein similarity detection leads naturally to algorithms operating at the metric, relational, and isotopic scales. The paper introduces a definition of similarity based on atomic motions that preserve local backbone topology without incurring significant distance errors. Such motions are motivated by the physical requirements for rearranging subsequences of a protein. Similarity detection then seeks rigid body motions able to overlay pairs of substructures, each related by a substructure-preserving motion, without necessarily requiring global structure preservation. This definition is general enough to span a wide range of questions: One can ask for full rearrangement of one protein into another while preserving global topology, as in drug design; or one can ask for rearrangements of sets of smaller substructures, preserving local but not global topology, as in protein evolution. In the appendix, we exhibit an algorithm for answering the general rearrangement question. That algorithm has the complexity of robot motion planning. In the text, we consider a more common case in which one seeks protein similarity by rearrangements of relatively short peptide segments. We exhibit two algorithms, one based on writhing numbers and one based on line weavings. The algorithms have time complexities O(n (4)) and O(s (11)), respectively, where n is the maximum number of residues in the proteins being compared and s is the number of secondary structure elements. In practice, the running times were nearly interactive. We report results obtained with a dozen pairs of proteins, exhibiting a range of typical features.