Original High-dimensional Data Research Articles

Joint Anchor Graph Embedding and Discrete Feature Scoring for Unsupervised Feature Selection.

The success of existing unsupervised feature selection (UFS) methods heavily relies on the assumption that the intrinsic relationships among original high-dimensional (HD) data samples exist in the discriminative low-dimension (LD) subspace. However, previous UFS methods commonly construct pairwise graphs and employ l2,1 -norm regularization to severally preserve the local structure and calculate the score of features, which is computationally complex and easy to get stuck into local optimum, so that those approaches cannot be applied in dealing with large-scale datasets in practice. To overcome this challenge, we propose a novel UFS method, in which a novel anchor graph embedding paradigm is designed to extract the local neighborhood relationships among data samples by reducing the computational complexity of graph construction to be linear in the number of data. Moreover, to improve the optimality of selected features as well as the performance of downstream tasks, we propose a discrete feature scoring mechanism, which imposes orthogonal l2,0 -norm constraints on learned projections, in order to enhance the distinction of feature scores as well as reduce the probability of falling into local optimum. In addition, solving the proposed nonconvex and nonsmooth NP-hard problem is challenging, and we present an efficient optimization algorithm to address it and acquire a closed-form solution of the transformation matrix. Extensive experiments demonstrate the effectiveness and efficiency of the proposed UFS by comparison with several state-of-the-art approaches to clustering and image segmentation tasks.

Tuning-free sparse clustering via alternating hard-thresholding

Model-based clustering is a commonly-used technique to partition heterogeneous data into homogeneous groups. When the analysis is to be conducted with a large number of features, analysts face simultaneous challenges in model interpretability, clustering accuracy, and computational efficiency. Several Bayesian and penalization methods have been proposed to select important features for model-based clustering. However, the performance of those methods relies on a careful algorithmic tuning, which can be time-consuming for high-dimensional cases. In this paper, we propose a new sparse clustering method based on alternating hard-thresholding. The new method is conceptually simple and tuning-free. With a user-specified sparsity level, it efficiently detects a set of key features by eliminating a large number of features that are less useful for clustering. Based on the selected key features, one can readily obtain an effective clustering of the original high-dimensional data under a general sparse covariance structure. Under mild conditions, we show that the new method leads to clusters with a misclassification rate consistent to the optimal rate as if the underlying true model were used. The promising performance of the new method is supported by both simulated and real data examples.

Optical fiber sensing and tensor AP clustering for high-speed road tunnel vehicle detection

Aiming at the problems of signal acquisition, feature extraction and vehicle recognition in highway tunnel vehicle detection, a new tunnel vehicle detection method is proposed by combining optical time-domain reflectometry distributed fiber sensing technology and tensor affine propagation clustering algorithm. Firstly, the distributed optical fiber system designed by optical time domain reflectometry was used to collect the running signals of tunnel vehicles and obtain the measurement data. Secondly, a high-order tensor sample set is constructed by using the spatial resolution of optical fiber as channel number and combining the feature number, time domain and frequency domain. Finally, tensor affine propagation clustering method and other clustering methods are used to test the accuracy. The test results show that the proposed method can better classify vehicles without destroying the original high-dimensional data structure. Meanwhile, the unsupervised clustering algorithm also reduces manual intervention in the identification process, increases the intelligence level of the whole vehicle detection model, and effectively improves the detection accuracy rate of tunnel vehicles.

Augmentation of degranulation mechanism for high-dimensional data with a multi-round optimization strategy

Various fuzzy clustering-based granulation–degranulation techniques have been developed for constructing and optimizing information granules, which help reveal the underlying structure of experimental data in Granular Computing (GrC). Basically, a well-performing granulation–degranulation mechanism runs with a low degranulation (reconstruction) error. However, the increasingly high-dimensional characteristics of data bring great challenges to achieve accurate of reconstruction of high-dimensional data. As such, for the reconstruction of high-dimensional data, an important issue is how to reduce the reconstruction error such that the data could be reconstructed more accurately. In order to address the challenge of unacceptable high reconstruction error posed by the increase in data dimensions and improve the inefficient fuzzy clustering-based granulation in existing techniques, this study develops a multi-round iterative optimization strategy with the use of Fuzzy C-Means (FCM) to enhance reconstruction performance for high-dimensional data. First, we propose a Feature Sampling-based FCM (FS-FCM) scheme served as the granulation mechanism in the framework. The proposed scheme draws on the idea of ensemble learning, where the granulation of original high-dimensional data is accomplished by generating and training low-dimensional sub-datasets through multiple times of feature random sampling. Then, a multi-round iterative granulation–degranulation mechanism is proposed along with its algorithmic framework. Within the proposed framework, we attempt to reduce the reconstruction error by iteratively reconstructing the residual data generated in each round of granulation–degranulation. Finally, we validate the developed strategy framework over twelve publicly available datasets with varying dimension scales. A set of ablation experiments verifies the effectiveness of the FS-FCM granulation scheme. The near-perfect reconstruction performance achieved by the proposed iterative framework on the given datasets further demonstrates its superiority.

Multiobjective ranking binary artificial bee colony for gene selection problems using microarray datasets

Microarray cancer gene expression datasets are high dimensional and complex for efficient computational analysis. Thus, selecting high discriminative genes from microarray data has become increasingly interesting in the field of bioinformatics. This article addresses the problem of designing a gene selection algorithm which is modeled as a multiobjective optimization problem with the sensitivity, specificity and the number of selected genes. Then, a multi-objective ranking binary artificial bee colony algorithm based on decomposition is proposed to select the optimal subset of dimensions from the original high dimensional data while retaining a subset that satisfies the defined objective. First, the Fisher-Markov selector is used to choose a fixed number of microarray data. Second, to make artificial bee colony algorithm suitable for the binary problem, a novel binary update strategy is proposed to balance the exploration and exploitation ability. Third, multi-objective ranking binary artificial bee colony algorithm is proposed by integrating tchebycheff approach into the ranking binary artificial bee colony algorithm. Finally, the multi-objective ranking binary artificial bee colony algorithm based on decomposition (MORBABC/D) method is used for feature selection, and extreme learning machine is used as the classifier with 10 fold cross-validation method. In order to show the effectiveness and efficiency of the algorithm, the proposed algorithm is tested on eight microarray dataset. Experimental studies have been carried out to investigate the ability of the proposed algorithm.

Joint Projection Learning and Tensor Decomposition-Based Incomplete Multiview Clustering.

Incomplete multiview clustering (IMVC) has received increasing attention since it is often that some views of samples are incomplete in reality. Most existing methods learn similarity subgraphs from original incomplete multiview data and seek complete graphs by exploring the incomplete subgraphs of each view for spectral clustering. However, the graphs constructed on the original high-dimensional data may be suboptimal due to feature redundancy and noise. Besides, previous methods generally ignored the graph noise caused by the interclass and intraclass structure variation during the transformation of incomplete graphs and complete graphs. To address these problems, we propose a novel joint projection learning and tensor decomposition (JPLTD)-based method for IMVC. Specifically, to alleviate the influence of redundant features and noise in high-dimensional data, JPLTD introduces an orthogonal projection matrix to project the high-dimensional features into a lower-dimensional space for compact feature learning. Meanwhile, based on the lower-dimensional space, the similarity graphs corresponding to instances of different views are learned, and JPLTD stacks these graphs into a third-order low-rank tensor to explore the high-order correlations across different views. We further consider the graph noise of projected data caused by missing samples and use a tensor-decomposition-based graph filter for robust clustering. JPLTD decomposes the original tensor into an intrinsic tensor and a sparse tensor. The intrinsic tensor models the true data similarities. An effective optimization algorithm is adopted to solve the JPLTD model. Comprehensive experiments on several benchmark datasets demonstrate that JPLTD outperforms the state-of-the-art methods. The code of JPLTD is available at https://github.com/weilvNJU/JPLTD.

Multigraph Fusion for Dynamic Graph Convolutional Network.

Graph convolutional network (GCN) outputs powerful representation by considering the structure information of the data to conduct representation learning, but its robustness is sensitive to the quality of both the feature matrix and the initial graph. In this article, we propose a novel multigraph fusion method to produce a high-quality graph and a low-dimensional space of original high-dimensional data for the GCN model. Specifically, the proposed method first extracts the common information and the complementary information among multiple local graphs to obtain a unified local graph, which is then fused with the global graph of the data to obtain the initial graph for the GCN model. As a result, the proposed method conducts the graph fusion process twice to simultaneously learn the low-dimensional space and the intrinsic graph structure of the data in a unified framework. Experimental results on real datasets demonstrated that our method outperformed the comparison methods in terms of classification tasks.

Locality Preserving Projections with Autoencoder

Locality Preserving Projections (LPP) is a popular dimensionality reduction method in the manifold learning field. However, LPP and all its variants only consider the one-way mapping from the high-dimensional space to the low-dimensional space and have no reverse verification, resulting in inaccurate low-dimensional embeddings. In this paper, we propose a new LPP method, called LPPAE (Locality Preserving Projections with Autoencoder), based on the linear Autoencoder. It constructs a two-way mapping: at the encoding stage, the conventional projection of LPP is viewed as a mapping from the high-dimensional space to the low-dimensional space. At the decoding stage, the low-dimensional embeddings are mapped back to the original high-dimensional space. The main contributions of the new method are: (1) This design not only preserves the neighborhood relationship of the data but more importantly, ensures that the low-dimensional embeddings can more accurately ”represent” the original data, thus significantly improving the performance of LPP. Experimental results on Handwritten Alphadigits, COIL-20, Yale, AR datasets show that the recognition rates of LPPAE are 26.06, 10.09, 5.40, and 8.86% higher than those of the original LPP respectively. On the MNIST dataset, compared to some of the latest improvements of LPP, including LPPMDC, LAPP, LPP+TR, and DNLPP, the recognition rate of LPPAE has been improved by 12.50, 38.10, 9.10, and 2.61%, respectively. (2) LPPAE regards the conventional LPP as an encoder, which is a new perspective. The idea of LPPAE can be used as a general framework and then extended to other manifold learning methods, and then a series of new methods can be developed.

Improving ISOMAP Efficiency with RKS: A Comparative Study with t-Distributed Stochastic Neighbor Embedding on Protein Sequences

Data visualization plays a crucial role in gaining insights from high-dimensional datasets. ISOMAP is a popular algorithm that maps high-dimensional data into a lower-dimensional space while preserving the underlying geometric structure. However, ISOMAP can be computationally expensive, especially for large datasets, due to the computation of the pairwise distances between data points. The motivation behind this study is to improve efficiency by leveraging an approximate method, which is based on random kitchen sinks (RKS). This approach provides a faster way to compute the kernel matrix. Using RKS significantly reduces the computational complexity of ISOMAP while still obtaining a meaningful low-dimensional representation of the data. We compare the performance of the approximate ISOMAP approach using RKS with the traditional t-SNE algorithm. The comparison involves computing the distance matrix using the original high-dimensional data and the low-dimensional data computed from both t-SNE and ISOMAP. The quality of the low-dimensional embeddings is measured using several metrics, including mean squared error (MSE), mean absolute error (MAE), and explained variance score (EVS). Additionally, the runtime of each algorithm is recorded to assess its computational efficiency. The comparison is conducted on a set of protein sequences, used in many bioinformatics tasks. We use three different embedding methods based on k-mers, minimizers, and position weight matrix (PWM) to capture various aspects of the underlying structure and the relationships between the protein sequences. By comparing different embeddings and by evaluating the effectiveness of the approximate ISOMAP approach using RKS and comparing it against t-SNE, we provide insights on the efficacy of our proposed approach. Our goal is to retain the quality of the low-dimensional embeddings while improving the computational performance.

Visual Exploration of Relationships and Structure in Low-Dimensional Embeddings.

In this work, we propose an interactive visual approach for the exploration and formation of structural relationships in embeddings of high-dimensional data. These structural relationships, such as item sequences, associations of items with groups, and hierarchies between groups of items, are defining properties of many real-world datasets. Nevertheless, most existing methods for the visual exploration of embeddings treat these structures as second-class citizens or do not take them into account at all. In our proposed analysis workflow, users explore enriched scatterplots of the embedding, in which relationships between items and/or groups are visually highlighted. The original high-dimensional data for single items, groups of items, or differences between connected items and groups is accessible through additional summary visualizations. We carefully tailored these summary and difference visualizations to the various data types and semantic contexts. During their exploratory analysis, users can externalize their insights by setting up additional groups and relationships between items and/or groups. We demonstrate the utility and potential impact of our approach by means of two use cases and multiple examples from various domains.

Joint Learning of Correlation-Constrained Fuzzy Clustering and Discriminative Non-Negative Representation for Hyperspectral Band Selection

Hyperspectral band selection plays an important role in overcoming the curse of dimensionality. Recently, clustering-based band selection methods have shown promise in the selection of informative and representative bands from hyperspectral images (HSIs). However, most existing clustering-based band selection methods involve the clustering of original HSIs, limiting their performance because of the high dimensionality of hyperspectral bands. To tackle this problem, a novel hyperspectral band selection method termed joint learning of correlation-constrained fuzzy clustering and discriminative non-negative representation for hyperspectral band selection (CFNR) is presented. In CFNR, graph regularized non-negative matrix factorization (GNMF) and constrained fuzzy C-means (FCM) are integrated into a unified model to perform clustering on the learned feature representation of bands rather than on the original high-dimensional data. Specifically, the proposed CFNR aims to learn the discriminative non-negative representation of each band for clustering by introducing GNMF into the model of the constrained FCM and making full use of the intrinsic manifold structure of HSIs. Moreover, based on the band correlation property of HSIs, a correlation constraint, which enforces the similarity of clustering results between neighboring bands, is imposed on the membership matrix of FCM in the CFNR model to obtain clustering results that meet the needs of band selection. The alternating direction multiplier method is adopted to solve the joint optimization model. Compared with existing methods, CFNR can obtain a more informative and representative band subset, thus can improve the reliability of hyperspectral image classifications. Experimental results on five real hyperspectral datasets demonstrate that CFNR can achieve superior performance compared with several state-of-the-art methods.

SpectralMAP: Approximating Data Manifold With Spectral Decomposition

Dimensionality reduction is widely used to visualize complex high-dimensional data. This study presents a novel method for effective data visualization. Previous methods depend on local distance measurements for data manifold approximation. This leads to unreliable results when a data manifold locally oscillates because of some undesirable effects, such as noise effects. In this study, we overcome this limitation by introducing a dual approximation of a data manifold. We roughly approximate a data manifold with a neighborhood graph and prune it with a global filter. This dual scheme results in local oscillation robustness and yields effective visualization with explicit global preservation. We consider a global filter based on principal component analysis frameworks and derive it with the spectral information of the original high-dimensional data. Finally, we experiment with multiple datasets to verify our method, compare its performance to that of state-of-the-art methods, and confirm the effectiveness of our novelty and results.

Provable General Bounded Denoisers for Snapshot Compressive Imaging With Convergence Guarantee

In the snapshot compressive imaging (SCI) field, how to explore priors for recovering the original high-dimensional data from its lower-dimensional measurements is a challenge. Recent plug-and-play efforts plugged by deep denoisers have achieved superior performance, and their convergences have been guaranteed under the assumption of bounded denoisers and the condition of diminishing noise levels. However, it is difficult to explicitly prove the bounded properties of existing deep denoisers due to complex network architectures. To address these issues, we propose a novel provable and trainable bounded denoiser using dual tight frames and spatial-variation thresholds. Furthermore, we combine the proposed trainable bounded denoiser with the well-known block matching 3D filtering (BM3D) denoiser for building a plug-and-play SCI framework. Precisely, we formulate a convergent plug-and-play SCI algorithm that can exploit complementary denoiser priors, and plug the two denoisers into it. We explicitly prove that these two denoisers are general bounded denoisers, and further show the convergence of the proposed SCI algorithm. Both simulation (video compressive imaging and hyperspectral compressive imaging) and real data results show that the proposed algorithm can achieve higher-quality reconstructions compared with benchmark SCI algorithms.

Hybrid-attention based Feature-reconstructive Adversarial Hashing Networks for Cross-modal Retrieval

With the massive growth of data of various modal types, people no longer use a single modal retrieval method, but a cross-modal retrieval method when performing retrieval tasks. Such methods often need to store data efficiently while maintaining the characteristics of fast query. Because the hashing learning method can represent the original high-dimensional data through a simple and compact binary hash code, which can greatly compress the data size and facilitate data storage and mutual retrieval, the cross-modal hashing retrieval has gradually become a hot topic in recent years. However, how to bridge the gap between modalities to improve the retrieval performance further is still a challenging problem. In order to solve this problem, we propose a Hybrid-attention based Feature-reconstructive Adversarial Hashing (HFAH) networks for cross-modal retrieval. First, a label semantic guidance module is introduced to guide the extraction process of text features and image features through the learning of labels, so as to fully maintain the semantic similarity between different modal data. Then, the hybrid-attention module is introduced to make the extracted data contain richer semantic information. Finally, the feature reconstruction network is used to make the relevant degree between similar cross-modal data pairs higher than that between dissimilar data pairs. Related experiments on two benchmark datasets confirm to us that HFAH performs better than several existing cross- modal retrieval methods.

Ridge regression coupled with a new uninformative variable elimination algorithm as a new descriptor screening method: Application of data reduction in QSAR study of some sulfonated derivatives as c-Met inhibitors

An uninformative variable elimination algorithm was combined with the Ridge regression method. This combination makes the penalized Ridge method, which is essentially incapable of deleting or selecting a variable, an efficient descriptor screening method for screening of descriptors in QSAR studies. The importance of descriptors was used to identify uninformative and redundant descriptors. The descriptor importance was defined as the ratio of the mean coefficients to the standard deviation of the coefficients derived from the leave-one-out (LOO) Ridge models. Then, with an iterative algorithm, among 392 molecular descriptors obtained after initial preprocessing, 358 uninformative descriptors were identified and removed from the data set. The remaining 34 descriptors were used for further variable selection and modeling through the smoothly clipped absolute deviation (SCAD) method. The proposed screening method was used to screen and reduce the size of QSAR data containing 63 compounds as c-Met inhibitors with anti-cancer activity. For comparison, the SCAD models were constructed using original high-dimensional data (SCAD), and the reduced-size data via the UVE-Ridge screening method (UVE-Ridge-SCAD). Both models were used to predict biological activities (pIC50) for compounds in the external test set. The coefficients of determination (R2) and the FIT parameter for the test data predicted using the SCAD model (with 12 selected descriptors) were equal to 0.8042 and 3.78, respectively. The UVE-Ridge-SCAD model (with 9 descriptors) showed R2 and FIT values of 0.9025 and 4.38, respectively. Higher R2 and FIT values indicate the excellent predictive power of the UVE-Ridge-SCAD model. Finally, the UVE-Ridge-SCAD model was used to suggest new high-potency compounds. The accuracy of the proposed compounds was investigated using ligand-receptor interactions.

A novel SSD fault detection method using GRU-based Sparse Auto-Encoder for dimensionality reduction

In recent years, with the development of flash memory technology, storage systems in large data centers are typically built upon thousands or even millions of solid-state drives (SSDs). Therefore, the failure of SSDs is inevitable. An SSD failure may cause unrecoverable data loss or unavailable system service, resulting in catastrophic results. Active fault detection technologies are able to detect device problems in advance, so it is gaining popularity. Recent trends have turned toward applying AI algorithms based on SSD SMART data for fault detection. However, SMART data of new SSDs contains a large number of features, and the high dimension of data features results in poor accuracy of AI algorithms for fault detection. To tackle the above problems, we improve the structure of traditional Auto-Encoder (AE) based on GRU and propose an SSD fault detection method – GAL based on dimensionality reduction with Gated Recurrent Unit (GRU) sparse autoencoder (GRUAE) by combining the temporal characteristics of SSD SMART data. The proposed method trains the GRUAE model with SSD SMART data firstly, and then adopts the encoder of GRUAE model as the dimensionality reduction tool to reduce the original high-dimensional SSD SMART data, aiming at reducing the influence of noise features in original SSD SAMRT data and highlight the features more relevant to data characteristics to improve the accuracy of fault detection. Finally, LSTM is adopted for fault detection with low-dimensional SSD SMART data. Experimental results on real SSD dataset from Alibaba show that the fault detection accuracy of various AI algorithms can be improved by varying degrees after dimensionality reduction with the proposed method, and GAL performs best among all methods.

An Efficient Method for Pricing Analysis Based on Neural Networks

The revolution in neural networks is a significant technological shift. It has an impact on not only all aspects of production and life, but also economic research. Neural networks have not only been a significant tool for economic study in recent years, but have also become an important topic of economics research, resulting in a large body of literature. The stock market is an important part of the country’s economic development, as well as our daily lives. Large dimensions and multiple collinearity characterize the stock index data. To minimize the number of dimensions in the data, multiple collinearity should be removed, and the stock price can then be forecast. To begin, a deep autoencoder based on the Restricted Boltzmann machine is built to encode high-dimensional input into low-dimensional space. Then, using a BP neural network, a regression model is created between low-dimensional coding sequence and stock price. The deep autoencoder’s capacity to extract this feature is superior to that of principal component analysis and factor analysis, according to the findings of the experiments. Utilizing the coded data, the proposed model can lower the computational cost and achieve higher prediction accuracy than using the original high-dimensional data.

A novel long sequence multi-step ship trajectory prediction method considering historical data

Considering the importance of timeliness in ship risk assessment, a high-resolution and long sequence multi-step trajectory prediction method is proposed. Through the multi-step prediction and uncertainty analysis of the ship trajectory for a long period of time, it is possible to make timely ship navigation risk assessment. First, the ship trajectory with high resolution is obtained by cubic spline interpolation. Then, Dynamic Time Warping (DTW) is used as the metric of distance, and a method called Laplacian Eigenmaps Self-Organizing Map (LE-SOM) is used to extract the features from original high-dimensional data with unequal intervals, so as to select the historical trajectories that can be used as a reference. Finally, the trajectory is generated for multi-step prediction. The proposed method not only predicts the position of the ship and its uncertainty from statistical perspective, but also investigates the relationship between trajectory curvature and the prediction error. The case study on a ferry ship in the Jiangsu section of the Yangtze River indicates the validity of the method.

Visual cluster separation using high-dimensional sharpened dimensionality reduction

Applying dimensionality reduction (DR) to large, high-dimensional data sets can be challenging when distinguishing the underlying high-dimensional data clusters in a 2D projection for exploratory analysis. We address this problem by first sharpening the clusters in the original high-dimensional data prior to the DR step using Local Gradient Clustering (LGC). We then project the sharpened data from the high-dimensional space to 2D by a user-selected DR method. The sharpening step aids this method to preserve cluster separation in the resulting 2D projection. With our method, end-users can label each distinct cluster to further analyze an otherwise unlabeled data set. Our “High-Dimensional Sharpened DR” (HD-SDR) method, tested on both synthetic and real-world data sets, is favorable to DR methods with poor cluster separation and yields a better visual cluster separation than these DR methods with no sharpening. Our method achieves good quality (measured by quality metrics) and scales computationally well with large high-dimensional data. To illustrate its concrete applications, we further apply HD-SDR on a recent astronomical catalog.

Hyperspectral Image Mixed Denoising Using Difference Continuity-Regularized Nonlocal Tensor Subspace Low-Rank Learning

With the rapid advancement of spectrometers, the imaging range of the electromagnetic spectrum starts growing narrower. The reduction of electromagnetic wave energy received in a single wavelength range leads more complex noise into the generated hyperspectral image (HSI), thus causing a severe cripple in the accuracy of subsequent applications. The requirement for the HSI mixed denoising algorithm’s accuracy is further lifted. To address this challenge, in this letter, we propose a novel difference continuity-regularized nonlocal tensor subspace low-rank learning (named DNTSLR) method for HSI mixed denoising. Technically, the original high-dimensional HSI data was first projected into a low-dimensional subspace spanned by a spectral difference continuous basis instead of an orthogonal basis, so the data continuity of the restored HSI spectrum and tensor low-rankness was guaranteed. Then, a cube matching strategy was employed to stack the nonlocal tensor patches from the projected coefficient tensor, and a shrinkage algorithm was used to approximate the low-rank coefficient tensor. Eventually, the subspace low-rank learning algorithm was designed to alternately separate the noise tensor and restore the latent clean low-rank HSI tensor. Extensive experiments on multiple open datasets validate that the proposed method realizes the state-of-the-art denoising accuracy for HSI.