Original High-dimensional Data Research Articles

Partially Observable Online Nonparametric Monitoring of Spatiotemporally Correlated Data Streams

Internet of Things sensor networks collect real-time data, characterized by spatial and temporal correlations, for process monitoring, significantly altering daily life and enabling automation. Considering sensor resource constraints due to limited budget of sensor operation and the complexity of capturing spatiotemporal correlation structure among data streams, sensor networks face challenges in monitoring such data streams via a parametric model and distribution, particularly when only subsets of data are available at each acquisition time. This paper develops a nonparametric scheme for monitoring such complex spatiotemporally correlated and partially observed data streams. It employs decorrelated rank-based statistics combined with data augmentation over multiple subdata streams, which are derived from the original high-dimensional data using ensemble random projections for dimensionality reduction. Monitoring and sampling decisions are informed by aggregated local statistics of all subdata streams. This method is distribution-free that eschews parametric spatiotemporal models and distributions for real-time monitoring, enhancing its practical applicability to various complex spatiotemporal engineering cases that cannot be accurately characterized by parametric models. The efficacy of the decorrelated rank-based statistics and sampling strategy is substantiated through theoretical analysis. Numerical experiments and case studies focusing on thermal data monitoring in grain storage and solar flare detection affirm robust performance of the proposed method across various scenarios.

Joint Projection Learning and Tensor Decomposition-Based Incomplete Multiview Clustering.

Incomplete multiview clustering (IMVC) has received increasing attention since it is often that some views of samples are incomplete in reality. Most existing methods learn similarity subgraphs from original incomplete multiview data and seek complete graphs by exploring the incomplete subgraphs of each view for spectral clustering. However, the graphs constructed on the original high-dimensional data may be suboptimal due to feature redundancy and noise. Besides, previous methods generally ignored the graph noise caused by the interclass and intraclass structure variation during the transformation of incomplete graphs and complete graphs. To address these problems, we propose a novel joint projection learning and tensor decomposition (JPLTD)-based method for IMVC. Specifically, to alleviate the influence of redundant features and noise in high-dimensional data, JPLTD introduces an orthogonal projection matrix to project the high-dimensional features into a lower-dimensional space for compact feature learning. Meanwhile, based on the lower-dimensional space, the similarity graphs corresponding to instances of different views are learned, and JPLTD stacks these graphs into a third-order low-rank tensor to explore the high-order correlations across different views. We further consider the graph noise of projected data caused by missing samples and use a tensor-decomposition-based graph filter for robust clustering. JPLTD decomposes the original tensor into an intrinsic tensor and a sparse tensor. The intrinsic tensor models the true data similarities. An effective optimization algorithm is adopted to solve the JPLTD model. Comprehensive experiments on several benchmark datasets demonstrate that JPLTD outperforms the state-of-the-art methods. The code of JPLTD is available at https://github.com/weilvNJU/JPLTD.

Weighted Fuzzy C-Means: Unsupervised Feature Selection to Realize a Target Partition

We introduce an unsupervised feature selection method based on regularized weighted Fuzzy C-Means (WRFCM) clustering. When the target task is clustering, our objective should be to select a subset of features that can generate the same/similar partition matrix to the partition matrix obtained from the original high dimensional data by a clustering algorithm. To achieve this we propose a novel objective function keeping in view the Fuzzy-C-Means (FCM) clustering algorithm. This approach realizes feature selection within the WRFCM framework, emphasizing features to maintain the FCM-based target partition. We evaluate our method using Normalized Mutual Information (NMI), Adjusted Rand Index (ARI) and Kuhn-Munkres index (KM-index). NMI, and ARI measure the agreement between clusters, i.e, the partition in the lower dimension and the partition of the original data. On the other hand, KM-index measures the disagreement between the two partitions. Experimental results on synthetic and real datasets showcase our method’s efficacy in selecting informative features. This approach fills a crucial gap in unsupervised feature selection, making it valuable for real-world applications. The approach is very general in the sense that the target partition can be generated by any clustering algorithm or even by the actual class labels of the data, when they are available.

Multi-view clustering with adaptive anchor and bipartite graph learning

Multi-view subspace clustering optimizes and integrates the graph structure information of each view. At present, the subspace clustering methods based on the abstract graph have better performance and improve the clustering results. Although the existing clustering algorithms have achieved excellent results, there are still four deficiencies: 1) Expensive time overhead, with most algorithms having a high time complexity; 2) Using fixed anchor points and the separation of anchor graph learning from subsequent graph construction, resulting in inadequate use of underlying graph structure between views; 3) Multi-view data have inevitable noise and in the original high-dimensional space data fusion may cause the loss of important information; 4) Most algorithms require additional post-processing to obtain clustering labels. To solve the above problems, we innovate a novel anchor-adaptive multi-view clustering algorithm based on a bipartite graph . Specifically, anchor graph learning and subspace graph construction are adaptively combined into a unified optimization framework. The projection matrix, consensus anchor matrix, and similarity matrix optimize each other to promote the clustering effect and structure a constrained bipartite graph to describe the relationship between representative points and sample points. At the same time, connectivity constraints are used to ensure that the connected components represent clusters directly. Our method has linear time complexity about sample size. Comparison experiments with the state-of-the-art algorithms demonstrate the effectiveness of the proposed method on various benchmark datasets. Moreover, our method is robust to noisy data and suitable for large-scale datasets.

A comprehensive learning based swarm optimization approach for feature selection in gene expression data

Gene expression data analysis is challenging due to the high dimensionality and complexity of the data. Feature selection, which identifies relevant genes, is a common preprocessing step. We propose a Comprehensive Learning-Based Swarm Optimization (CLBSO) approach for feature selection in gene expression data. CLBSO leverages the strengths of ants and grasshoppers to efficiently explore the high-dimensional search space. Ants perform local search and leave pheromone trails to guide the swarm, while grasshoppers use their ability to jump long distances to explore new regions and avoid local optima. The proposed approach was evaluated on several publicly available gene expression datasets and compared with state-of-the-art feature selection methods. CLBSO achieved an average accuracy improvement of 15% over the original high-dimensional data and outperformed other feature selection methods by up to 10%. For instance, in the Pancreatic cancer dataset, CLBSO achieved 97.2% accuracy, significantly higher than XGBoost-MOGA's 84.0%. Convergence analysis showed CLBSO required fewer iterations to reach optimal solutions. Statistical analysis confirmed significant performance improvements, and stability analysis demonstrated consistent gene subset selection across different runs. These findings highlight the robustness and efficacy of CLBSO in handling complex gene expression datasets, making it a valuable tool for enhancing classification tasks in bioinformatics.

Comprehensive consensus representation learning for incomplete multiview subspace clustering

Incomplete multiview clustering (IMVC) aims to address clustering problems caused by missing views. While some progress has been made, the current approaches still have some problems. First, the existing IMVC methods excessively rely on the original high-dimensional multiview data, leading to suboptimal results when the data are heavily contaminated. Second, the utilization of high-dimensional data often overlooks the complementarity of and consistency across multiple views. Third, the cluster structure of the input data is not adequately considered, which is detrimental to the resulting clustering performance. To address these problems, we propose a novel method: comprehensive consensus representation learning for incomplete multiview subspace clustering (CLR-IMVC). Specifically, CLR-IMVC assumes that all views are generated from a latent representation space. We model the original incomplete multiple views and the latent space to explore their correlations. Subsequently, we use the latent space to generate complete views and construct a subspace self-representation matrix for each view. Moreover, we innovatively treat the latent representation itself as new a view and likewise extract its self-representation matrix. Furthermore, we construct a third-order tensor by using these self-representation matrices and impose low-rank constraints on the tensor to explore the high-order correlations among the matrices. In addition, a consensus representation learning term is added to explore the consistency information between the views and the latent space, ultimately generating a robust clustering structure. Finally, the proposed method can be effectively solved by the augmented Lagrangian multiplier (ALM). Extensive experiments conducted on diverse datasets demonstrate the effectiveness of CLR-IMVC.

Joint Anchor Graph Embedding and Discrete Feature Scoring for Unsupervised Feature Selection.

The success of existing unsupervised feature selection (UFS) methods heavily relies on the assumption that the intrinsic relationships among original high-dimensional (HD) data samples exist in the discriminative low-dimension (LD) subspace. However, previous UFS methods commonly construct pairwise graphs and employ l2,1 -norm regularization to severally preserve the local structure and calculate the score of features, which is computationally complex and easy to get stuck into local optimum, so that those approaches cannot be applied in dealing with large-scale datasets in practice. To overcome this challenge, we propose a novel UFS method, in which a novel anchor graph embedding paradigm is designed to extract the local neighborhood relationships among data samples by reducing the computational complexity of graph construction to be linear in the number of data. Moreover, to improve the optimality of selected features as well as the performance of downstream tasks, we propose a discrete feature scoring mechanism, which imposes orthogonal l2,0 -norm constraints on learned projections, in order to enhance the distinction of feature scores as well as reduce the probability of falling into local optimum. In addition, solving the proposed nonconvex and nonsmooth NP-hard problem is challenging, and we present an efficient optimization algorithm to address it and acquire a closed-form solution of the transformation matrix. Extensive experiments demonstrate the effectiveness and efficiency of the proposed UFS by comparison with several state-of-the-art approaches to clustering and image segmentation tasks.

Tuning-free sparse clustering via alternating hard-thresholding

Model-based clustering is a commonly-used technique to partition heterogeneous data into homogeneous groups. When the analysis is to be conducted with a large number of features, analysts face simultaneous challenges in model interpretability, clustering accuracy, and computational efficiency. Several Bayesian and penalization methods have been proposed to select important features for model-based clustering. However, the performance of those methods relies on a careful algorithmic tuning, which can be time-consuming for high-dimensional cases. In this paper, we propose a new sparse clustering method based on alternating hard-thresholding. The new method is conceptually simple and tuning-free. With a user-specified sparsity level, it efficiently detects a set of key features by eliminating a large number of features that are less useful for clustering. Based on the selected key features, one can readily obtain an effective clustering of the original high-dimensional data under a general sparse covariance structure. Under mild conditions, we show that the new method leads to clusters with a misclassification rate consistent to the optimal rate as if the underlying true model were used. The promising performance of the new method is supported by both simulated and real data examples.

Optical fiber sensing and tensor AP clustering for high-speed road tunnel vehicle detection

Aiming at the problems of signal acquisition, feature extraction and vehicle recognition in highway tunnel vehicle detection, a new tunnel vehicle detection method is proposed by combining optical time-domain reflectometry distributed fiber sensing technology and tensor affine propagation clustering algorithm. Firstly, the distributed optical fiber system designed by optical time domain reflectometry was used to collect the running signals of tunnel vehicles and obtain the measurement data. Secondly, a high-order tensor sample set is constructed by using the spatial resolution of optical fiber as channel number and combining the feature number, time domain and frequency domain. Finally, tensor affine propagation clustering method and other clustering methods are used to test the accuracy. The test results show that the proposed method can better classify vehicles without destroying the original high-dimensional data structure. Meanwhile, the unsupervised clustering algorithm also reduces manual intervention in the identification process, increases the intelligence level of the whole vehicle detection model, and effectively improves the detection accuracy rate of tunnel vehicles.

Augmentation of degranulation mechanism for high-dimensional data with a multi-round optimization strategy

Various fuzzy clustering-based granulation–degranulation techniques have been developed for constructing and optimizing information granules, which help reveal the underlying structure of experimental data in Granular Computing (GrC). Basically, a well-performing granulation–degranulation mechanism runs with a low degranulation (reconstruction) error. However, the increasingly high-dimensional characteristics of data bring great challenges to achieve accurate of reconstruction of high-dimensional data. As such, for the reconstruction of high-dimensional data, an important issue is how to reduce the reconstruction error such that the data could be reconstructed more accurately. In order to address the challenge of unacceptable high reconstruction error posed by the increase in data dimensions and improve the inefficient fuzzy clustering-based granulation in existing techniques, this study develops a multi-round iterative optimization strategy with the use of Fuzzy C-Means (FCM) to enhance reconstruction performance for high-dimensional data. First, we propose a Feature Sampling-based FCM (FS-FCM) scheme served as the granulation mechanism in the framework. The proposed scheme draws on the idea of ensemble learning, where the granulation of original high-dimensional data is accomplished by generating and training low-dimensional sub-datasets through multiple times of feature random sampling. Then, a multi-round iterative granulation–degranulation mechanism is proposed along with its algorithmic framework. Within the proposed framework, we attempt to reduce the reconstruction error by iteratively reconstructing the residual data generated in each round of granulation–degranulation. Finally, we validate the developed strategy framework over twelve publicly available datasets with varying dimension scales. A set of ablation experiments verifies the effectiveness of the FS-FCM granulation scheme. The near-perfect reconstruction performance achieved by the proposed iterative framework on the given datasets further demonstrates its superiority.

Application of Dimension Reduction Methods to High-Dimensional Single-Cell 3D Genomic Contact Data

The volume and complexity of data in various fields, particularly in biology, are increasing exponentially, posing a challenge to existing analytical methods, which often struggle with high-dimensional data such as single-cell Hi-C data. To address this issue, we employ unsupervised methods, specifically Principal Component Analysis (PCA) and t-Distributed Stochastic Neighbor Embedding (t-SNE), to reduce data dimensions for visualization. Furthermore, we assess the information retention of the decomposed components using a Linear Discriminant Analysis (LDA) classifier model. Our findings indicate that these dimensionality reduction techniques effectively capture and present information not readily apparent in the original high-dimensional data, facilitating the visualization and interpretation of complex biological data. The LDA classifier's performance suggests that PCA and t-SNE maintain critical information necessary for accurate classification. In conclusion, our study demonstrates that PCA and t-SNE are powerful tools for visualizing and analyzing high-dimensional biological data, enabling researchers to gain new insights and understandings that are challenging to achieve with traditional approaches.

Multiobjective ranking binary artificial bee colony for gene selection problems using microarray datasets

Microarray cancer gene expression datasets are high dimensional and complex for efficient computational analysis. Thus, selecting high discriminative genes from microarray data has become increasingly interesting in the field of bioinformatics. This article addresses the problem of designing a gene selection algorithm which is modeled as a multiobjective optimization problem with the sensitivity, specificity and the number of selected genes. Then, a multi-objective ranking binary artificial bee colony algorithm based on decomposition is proposed to select the optimal subset of dimensions from the original high dimensional data while retaining a subset that satisfies the defined objective. First, the Fisher-Markov selector is used to choose a fixed number of microarray data. Second, to make artificial bee colony algorithm suitable for the binary problem, a novel binary update strategy is proposed to balance the exploration and exploitation ability. Third, multi-objective ranking binary artificial bee colony algorithm is proposed by integrating tchebycheff approach into the ranking binary artificial bee colony algorithm. Finally, the multi-objective ranking binary artificial bee colony algorithm based on decomposition (MORBABC/D) method is used for feature selection, and extreme learning machine is used as the classifier with 10 fold cross-validation method. In order to show the effectiveness and efficiency of the algorithm, the proposed algorithm is tested on eight microarray dataset. Experimental studies have been carried out to investigate the ability of the proposed algorithm.

Multigraph Fusion for Dynamic Graph Convolutional Network.

Graph convolutional network (GCN) outputs powerful representation by considering the structure information of the data to conduct representation learning, but its robustness is sensitive to the quality of both the feature matrix and the initial graph. In this article, we propose a novel multigraph fusion method to produce a high-quality graph and a low-dimensional space of original high-dimensional data for the GCN model. Specifically, the proposed method first extracts the common information and the complementary information among multiple local graphs to obtain a unified local graph, which is then fused with the global graph of the data to obtain the initial graph for the GCN model. As a result, the proposed method conducts the graph fusion process twice to simultaneously learn the low-dimensional space and the intrinsic graph structure of the data in a unified framework. Experimental results on real datasets demonstrated that our method outperformed the comparison methods in terms of classification tasks.

Locality Preserving Projections with Autoencoder

Locality Preserving Projections (LPP) is a popular dimensionality reduction method in the manifold learning field. However, LPP and all its variants only consider the one-way mapping from the high-dimensional space to the low-dimensional space and have no reverse verification, resulting in inaccurate low-dimensional embeddings. In this paper, we propose a new LPP method, called LPPAE (Locality Preserving Projections with Autoencoder), based on the linear Autoencoder. It constructs a two-way mapping: at the encoding stage, the conventional projection of LPP is viewed as a mapping from the high-dimensional space to the low-dimensional space. At the decoding stage, the low-dimensional embeddings are mapped back to the original high-dimensional space. The main contributions of the new method are: (1) This design not only preserves the neighborhood relationship of the data but more importantly, ensures that the low-dimensional embeddings can more accurately ”represent” the original data, thus significantly improving the performance of LPP. Experimental results on Handwritten Alphadigits, COIL-20, Yale, AR datasets show that the recognition rates of LPPAE are 26.06, 10.09, 5.40, and 8.86% higher than those of the original LPP respectively. On the MNIST dataset, compared to some of the latest improvements of LPP, including LPPMDC, LAPP, LPP+TR, and DNLPP, the recognition rate of LPPAE has been improved by 12.50, 38.10, 9.10, and 2.61%, respectively. (2) LPPAE regards the conventional LPP as an encoder, which is a new perspective. The idea of LPPAE can be used as a general framework and then extended to other manifold learning methods, and then a series of new methods can be developed.

Improving ISOMAP Efficiency with RKS: A Comparative Study with t-Distributed Stochastic Neighbor Embedding on Protein Sequences

Data visualization plays a crucial role in gaining insights from high-dimensional datasets. ISOMAP is a popular algorithm that maps high-dimensional data into a lower-dimensional space while preserving the underlying geometric structure. However, ISOMAP can be computationally expensive, especially for large datasets, due to the computation of the pairwise distances between data points. The motivation behind this study is to improve efficiency by leveraging an approximate method, which is based on random kitchen sinks (RKS). This approach provides a faster way to compute the kernel matrix. Using RKS significantly reduces the computational complexity of ISOMAP while still obtaining a meaningful low-dimensional representation of the data. We compare the performance of the approximate ISOMAP approach using RKS with the traditional t-SNE algorithm. The comparison involves computing the distance matrix using the original high-dimensional data and the low-dimensional data computed from both t-SNE and ISOMAP. The quality of the low-dimensional embeddings is measured using several metrics, including mean squared error (MSE), mean absolute error (MAE), and explained variance score (EVS). Additionally, the runtime of each algorithm is recorded to assess its computational efficiency. The comparison is conducted on a set of protein sequences, used in many bioinformatics tasks. We use three different embedding methods based on k-mers, minimizers, and position weight matrix (PWM) to capture various aspects of the underlying structure and the relationships between the protein sequences. By comparing different embeddings and by evaluating the effectiveness of the approximate ISOMAP approach using RKS and comparing it against t-SNE, we provide insights on the efficacy of our proposed approach. Our goal is to retain the quality of the low-dimensional embeddings while improving the computational performance.

Visual Exploration of Relationships and Structure in Low-Dimensional Embeddings.

In this work, we propose an interactive visual approach for the exploration and formation of structural relationships in embeddings of high-dimensional data. These structural relationships, such as item sequences, associations of items with groups, and hierarchies between groups of items, are defining properties of many real-world datasets. Nevertheless, most existing methods for the visual exploration of embeddings treat these structures as second-class citizens or do not take them into account at all. In our proposed analysis workflow, users explore enriched scatterplots of the embedding, in which relationships between items and/or groups are visually highlighted. The original high-dimensional data for single items, groups of items, or differences between connected items and groups are accessible through additional summary visualizations. We carefully tailored these summary and difference visualizations to the various data types and semantic contexts. During their exploratory analysis, users can externalize their insights by setting up additional groups and relationships between items and/or groups. We demonstrate the utility and potential impact of our approach by means of two use cases and multiple examples from various domains.

Joint Learning of Correlation-Constrained Fuzzy Clustering and Discriminative Non-Negative Representation for Hyperspectral Band Selection

Hyperspectral band selection plays an important role in overcoming the curse of dimensionality. Recently, clustering-based band selection methods have shown promise in the selection of informative and representative bands from hyperspectral images (HSIs). However, most existing clustering-based band selection methods involve the clustering of original HSIs, limiting their performance because of the high dimensionality of hyperspectral bands. To tackle this problem, a novel hyperspectral band selection method termed joint learning of correlation-constrained fuzzy clustering and discriminative non-negative representation for hyperspectral band selection (CFNR) is presented. In CFNR, graph regularized non-negative matrix factorization (GNMF) and constrained fuzzy C-means (FCM) are integrated into a unified model to perform clustering on the learned feature representation of bands rather than on the original high-dimensional data. Specifically, the proposed CFNR aims to learn the discriminative non-negative representation of each band for clustering by introducing GNMF into the model of the constrained FCM and making full use of the intrinsic manifold structure of HSIs. Moreover, based on the band correlation property of HSIs, a correlation constraint, which enforces the similarity of clustering results between neighboring bands, is imposed on the membership matrix of FCM in the CFNR model to obtain clustering results that meet the needs of band selection. The alternating direction multiplier method is adopted to solve the joint optimization model. Compared with existing methods, CFNR can obtain a more informative and representative band subset, thus can improve the reliability of hyperspectral image classifications. Experimental results on five real hyperspectral datasets demonstrate that CFNR can achieve superior performance compared with several state-of-the-art methods.

SpectralMAP: Approximating Data Manifold With Spectral Decomposition

Dimensionality reduction is widely used to visualize complex high-dimensional data. This study presents a novel method for effective data visualization. Previous methods depend on local distance measurements for data manifold approximation. This leads to unreliable results when a data manifold locally oscillates because of some undesirable effects, such as noise effects. In this study, we overcome this limitation by introducing a dual approximation of a data manifold. We roughly approximate a data manifold with a neighborhood graph and prune it with a global filter. This dual scheme results in local oscillation robustness and yields effective visualization with explicit global preservation. We consider a global filter based on principal component analysis frameworks and derive it with the spectral information of the original high-dimensional data. Finally, we experiment with multiple datasets to verify our method, compare its performance to that of state-of-the-art methods, and confirm the effectiveness of our novelty and results.

Provable General Bounded Denoisers for Snapshot Compressive Imaging With Convergence Guarantee

In the snapshot compressive imaging (SCI) field, how to explore priors for recovering the original high-dimensional data from its lower-dimensional measurements is a challenge. Recent plug-and-play efforts plugged by deep denoisers have achieved superior performance, and their convergences have been guaranteed under the assumption of bounded denoisers and the condition of diminishing noise levels. However, it is difficult to explicitly prove the bounded properties of existing deep denoisers due to complex network architectures. To address these issues, we propose a novel provable and trainable bounded denoiser using dual tight frames and spatial-variation thresholds. Furthermore, we combine the proposed trainable bounded denoiser with the well-known block matching 3D filtering (BM3D) denoiser for building a plug-and-play SCI framework. Precisely, we formulate a convergent plug-and-play SCI algorithm that can exploit complementary denoiser priors, and plug the two denoisers into it. We explicitly prove that these two denoisers are general bounded denoisers, and further show the convergence of the proposed SCI algorithm. Both simulation (video compressive imaging and hyperspectral compressive imaging) and real data results show that the proposed algorithm can achieve higher-quality reconstructions compared with benchmark SCI algorithms.

Hybrid-attention based Feature-reconstructive Adversarial Hashing Networks for Cross-modal Retrieval

With the massive growth of data of various modal types, people no longer use a single modal retrieval method, but a cross-modal retrieval method when performing retrieval tasks. Such methods often need to store data efficiently while maintaining the characteristics of fast query. Because the hashing learning method can represent the original high-dimensional data through a simple and compact binary hash code, which can greatly compress the data size and facilitate data storage and mutual retrieval, the cross-modal hashing retrieval has gradually become a hot topic in recent years. However, how to bridge the gap between modalities to improve the retrieval performance further is still a challenging problem. In order to solve this problem, we propose a Hybrid-attention based Feature-reconstructive Adversarial Hashing (HFAH) networks for cross-modal retrieval. First, a label semantic guidance module is introduced to guide the extraction process of text features and image features through the learning of labels, so as to fully maintain the semantic similarity between different modal data. Then, the hybrid-attention module is introduced to make the extracted data contain richer semantic information. Finally, the feature reconstruction network is used to make the relevant degree between similar cross-modal data pairs higher than that between dissimilar data pairs. Related experiments on two benchmark datasets confirm to us that HFAH performs better than several existing cross- modal retrieval methods.