High-dimensional Samples Research Articles

Decision system for copper flotation backbone process

This study proposed a decision system that can output the flotation backbone flowchart using the natural properties of copper ore. The proposed decision system includes three decision tasks: product scheme, flotation scheme and grinding scheme. Each decision task is a multi-label classification problem. To improve the classification effect of each sub-label, extreme gradient boosting (XGBoost) is used as a subclassifier, because of its ability to deal with small and high-dimensional samples. To selectively utilize the relations between the sub-labels in the same task, a modified classifier chain (MCC) was proposed. To specifically use the effect of a front-end task on a back-end task, the decision system connects the MCC-XGBoost corresponding to the three tasks in series. Accordingly, the outputs of a front-end task becomes the candidate features of a back-end task. To improve the recall rates of minority classes, the classification thresholds are customized using the Yoden index. Finally, the high performance of the decision system was demonstrated by hypothesis testing.

Central limit theorem of linear spectral statistics of high-dimensional sample correlation matrices

A high-dimensional sample correlation matrix is an important random matrix in multivariate statistical analysis. Its central limit theory is one of the main theoretical bases for making statistical inferences on high-dimensional correlation matrices. Under the high-dimensional framework in which the data dimension tends to infinity proportionally with the sample size, we establish the central limit theorems (CLT) for the linear spectral statistics (LSS) of sample correlation matrices in two settings: (1) the population follows an independent component structure; (2) the population follows an elliptical structure, including some heavy-tailed distributions. The results show that the CLTs of the LSS of the sample correlation matrices are very different in the two settings. In particular, even if the population correlation matrix is an identity matrix, the CLTs are different in the two settings. An application of our two established CLTs is provided.

A manifold two-sample test study: integral probability metric with neural networks

Abstract Two-sample tests are important areas aiming to determine whether two collections of observations follow the same distribution or not. We propose two-sample tests based on integral probability metric (IPM) for high-dimensional samples supported on a low-dimensional manifold. We characterize the properties of proposed tests with respect to the number of samples $n$ and the structure of the manifold with intrinsic dimension $d$. When an atlas is given, we propose a two-step test to identify the difference between general distributions, which achieves the type-II risk in the order of $n^{-1/\max \{d,2\}}$. When an atlas is not given, we propose Hölder IPM test that applies for data distributions with $(s,\beta )$-Hölder densities, which achieves the type-II risk in the order of $n^{-(s+\beta )/d}$. To mitigate the heavy computation burden of evaluating the Hölder IPM, we approximate the Hölder function class using neural networks. Based on the approximation theory of neural networks, we show that the neural network IPM test has the type-II risk in the order of $n^{-(s+\beta )/d}$, which is in the same order of the type-II risk as the Hölder IPM test. Our proposed tests are adaptive to low-dimensional geometric structure because their performance crucially depends on the intrinsic dimension instead of the data dimension.

Supervised feature selection on gene expression microarray datasets using manifold learning

In recent decades, the ultimate output from microarray assay, has produced enormous numbers of microarray datasets, regardless of the used technology. These datasets include complex and high dimensional samples and genes that the number of samples is much smaller than the number of genes (features). Due to the redundant dimensions in these datasets, processing them directly not only leads to poor performance but also increases computation time and memory usage. Feature selection reduces computational expense while improving or maintaining diagnosis accuracy. In this study, we propose a new supervised feature selection method based on a manifold learning approach. We focus in two different directions to address this issue. First, maximum relevancy criterion that achieves by integrating Supervised Laplacian Eigenmaps (S-LE) and a matrix, which can realize the process of feature selection. The applied criterion simultaneously opts the features that make same-class samples closer to each other and ignores the features that cause different-class samples be near. Second, minimum redundancy among selected features by applying the Pearson correlation coefficient. In the test phase, the proposed method is compared with ten state-of-the-art algorithms on seven microarray datasets. Reported results show that the proposed method has more promising performance than the other methods.

Coal Spray Rate Prediction Based on Factor Analysis and Neural Network (NN) Algorithm

Given the many factors that affect coal bursts as well as complex nonlinear relationships, this study analyzes the main factors that affect coal bursts in coal seams. Central Sulawesi Province. Using NN algorithms to predict coal bursts. This algorithm uses factor analysis to subtract the attributes of the original high-dimensional sample and obtains three common factors to maintain the correlation characteristics of 82.227% of the original data that had 10 sets. factors affecting coal burst rates can be used as training datasets as well as the last five datasets used as testing data and inputted into the rapid miner application to support NN algorithms to predict results. By comparing the prediction results where the NN algorithm has a non-explosion threshold value of 3.194% and an explosion threshold value of -3.230% and is more suitable for predicting coal burst explosions.

A Bayesian Semisupervised Approach to Keyword Extraction with Only Positive and Unlabeled Data

In the era of big data, people benefit from the existence of tremendous amounts of information. However, availability of said information may pose great challenges. For instance, one big challenge is how to extract useful yet succinct information in an automated fashion. As one of the first few efforts, keyword extraction methods summarize an article by identifying a list of keywords. Many existing keyword extraction methods focus on the unsupervised setting, with all keywords assumed unknown. In reality, a (small) subset of the keywords may be available for a particular article. To use such information, we propose a rigorous probabilistic model based on a semisupervised setup. Our method incorporates the graph-based information of an article into a Bayesian framework via an informative prior so that our model facilitates formal statistical inference, which is often absent from existing methods. To overcome the difficulty arising from high-dimensional posterior sampling, we develop two Markov chain Monte Carlo algorithms based on Gibbs samplers and compare their performance using benchmark data. We use a false discovery rate (FDR)-based approach for selecting the number of keywords, whereas the existing methods use ad hoc threshold values. Our numerical results show that the proposed method compared favorably with state-of-the-art methods for keyword extraction. History: Accepted by Ramaswamy Ramesh, Area Editor for Data Science and Machine Learning. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1283 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0234 ) at ( http://dx.doi.org/10.5281/zenodo.7348935 ).

A Self-Supervised Fault Detection for UAV Based on Unbalanced Flight Data Representation Learning and Wavelet Analysis

This paper aims to build a Self-supervised Fault Detection Model for UAVs combined with an Auto-Encoder. With the development of data science, it is imperative to detect UAV faults and improve their safety. Many factors affect the fault of a UAV, such as the voltage of the generator, angle of attack, and position of the rudder surface. A UAV is a typical complex system, and its flight data are typical high-dimensional large sample data sets. In practical applications such as UAV fault detection, the fault data only appear in a small part of the data sets. In this study, representation learning is used to extract the normal features of the flight data and reduce the dimensions of the data. The normal data are used for the training of the Auto-Encoder, and the reconstruction loss is used as the criterion for fault detection. An Improved Auto-Encoder suitable for UAV Flight Data Sets is proposed in this paper. In the Auto-Encoder, we use wavelet analysis to extract the low-frequency signals with different frequencies from the flight data. The Auto-Encoder is used for the feature extraction and reconstruction of the low-frequency signals with different frequencies. To improve the effectiveness of the fault localization at inference, we develop a new fault factor location model, which is based on the reconstruction loss of the Auto-Encoder and edge detection operator. The UAV Flight Data Sets are used for hard-landing detection, and an average accuracy of 91.01% is obtained. Compared with other models, the results suggest that the developed Self-supervised Fault Detection Model for UAVs has better accuracy. Concluding this study, an explanation is provided concerning the proposed model’s good results.

Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencoder.

In this paper, we propose a new method to perform data augmentation in a reliable way in the High Dimensional Low Sample Size (HDLSS) setting using a geometry-based variational autoencoder (VAE). Our approach combines the proposal of 1) a new VAE model, the latent space of which is modeled as a Riemannian manifold and which combines both Riemannian metric learning and normalizing flows and 2) a new generation scheme which produces more meaningful samples especially in the context of small data sets. The method is tested through a wide experimental study where its robustness to data sets, classifiers and training samples size is stressed. It is also validated on a medical imaging classification task on the challenging ADNI database where a small number of 3D brain magnetic resonance images (MRIs) are considered and augmented using the proposed VAE framework. In each case, the proposed method allows for a significant and reliable gain in the classification metrics. For instance, balanced accuracy jumps from 66.3% to 74.3% for a state-of-the-art convolutional neural network classifier trained with 50 MRIs of cognitively normal (CN) and 50 Alzheimer disease (AD) patients and from 77.7% to 86.3% when trained with 243 CN and 210 AD while improving greatly sensitivity and specificity metrics.

Convergence rate to the Tracy–Widom laws for the largest eigenvalue of sample covariance matrices

We establish a quantitative version of the Tracy–Widom law for the largest eigenvalue of high-dimensional sample covariance matrices. To be precise, we show that the fluctuations of the largest eigenvalue of a sample covariance matrix X∗X converge to its Tracy–Widom limit at a rate nearly N−1/3, where X is an M×N random matrix whose entries are independent real or complex random variables, assuming that both M and N tend to infinity at a constant rate. This result improves the previous estimate N−2/9 obtained by Wang (2019). Our proof relies on a Green function comparison method (Adv. Math. 229 (2012) 1435–1515) using iterative cumulant expansions, the local laws for the Green function and asymptotic properties of the correlation kernel of the white Wishart ensemble.

SPNE: sample-perturbed network entropy for revealing critical states of complex biological systems.

Complex biological systems do not always develop smoothly but occasionally undergo a sharp transition; i.e. there exists a critical transition or tipping point at which a drastic qualitative shift occurs. Hunting for such a critical transition is important to prevent or delay the occurrence of catastrophic consequences, such as disease deterioration. However, the identification of the critical state for complex biological systems is still a challenging problem when using high-dimensional small sample data, especially where only a certain sample is available, which often leads to the failure of most traditional statistical approaches. In this study, a novel quantitative method, sample-perturbed network entropy (SPNE), is developed based on the sample-perturbed directed network to reveal the critical state of complex biological systems at the single-sample level. Specifically, the SPNE approach effectively quantifies the perturbation effect caused by a specific sample on the directed network in terms of network entropy and thus captures the criticality of biological systems. This model-free method was applied to both bulk and single-cell expression data. Our approach was validated by successfully detecting the early warning signals of the critical states for six real datasets, including four tumor datasets from The Cancer Genome Atlas (TCGA) and two single-cell datasets of cell differentiation. In addition, the functional analyses of signaling biomarkers demonstrated the effectiveness of the analytical and computational results.

An Efficient Hyperspectral Image Classification Method Using Deep Fusion of 3-D Discrete Wavelet Transform and CNN

For the predominant classification performance, the convolutional neural network (CNN) is becoming quite popular in hyperspectral image (HSI) classification. However, quite a few parameters must be updated in the training procedure. The overfitting problem is also exacerbated by HSI’s high dimensionality and limited training samples. Therefore, an end-to-end model DWTCNN that deeply integrates 3-D discrete wavelet transform (DWT) and CNN is proposed. The 3-D DWT is introduced for the intrinsic feature collection in multi-resolution, and the 3-D DWT modulated kernel with predefined parameters will not increase the trainable parameters of DWTCNN. The impact of 3D-DWT at different scale levels on classification performance is also studied. The amalgamation of 3-D DWT reduces the feature extraction burden of 3-D CNN while increasing the feature extraction capacity of the network. With a compact and light structure, our model is easier to train with less memory and limited training samples. Compared with other methods, our model has a considerable advantage in computational efficiency while maintaining good classification accuracy and clear physical meaning interpretability. Experimental results on three public datasets also demonstrate the superiority of our approach in the training process and classification performance.

Learning Quantum Distributions with Variational Diffusion Models

It is challenging to identify the state of many-body quantum systems, as recovering density matrices underlying the quantum state typically requires computational resources scale exponentially to the system size. In this work, we introduce the variational diffusion model (VDM) to efficiently learn high-dimensional quantum distributions with high fidelity, which is essential to realize the fast reconstruction of quantum states. We build up the VDM suitable for dealing with the high-dimensional quantum samples, and then perform numerical experiments to test our model and other autoregressive models, including recurrent neural network and transformer. It is found that the VDM can achieve a modest better performance with fewest parameters than other two to learn the distribution as desired. Our results pave the way to applying diffusion models to solve hard problems in the quantum domain.

A Novel Density Peaks Clustering Algorithm with Isolation Kernel and K-Induction

Density peaks clustering (DPC) algorithm can process data of any shape and is simple and intuitive. However, the distance between any two high-dimensional points tends to be consistent, which makes it difficult to distinguish the density peaks and easily produces “bad label” delivery. To surmount the above-mentioned defects, this paper put forward a novel density peaks clustering algorithm with isolation kernel and K-induction (IKDC). The IKDC uses an optimized isolation kernel instead of the traditional distance. The optimized isolation kernel solves the problem of converging the distance between the high-dimensional samples by increasing the similarity of two samples in a sparse domain and decreasing the similarity of two samples in a dense domain. In addition, the IKDC introduces three-way clustering, uses core domains to represent dense regions of clusters, and uses boundary domains to represent sparse regions of clusters, where points in the boundary domains may belong to one or more clusters. At the same time as determining the core domains, the improved KNN and average similarity are proposed to assign as many as possible to the core domains. The K-induction is proposed to assign the leftover points to the boundary domain of the optimal cluster. To confirm the practicability and validity of IKDC, we test on 10 synthetic and 8 real datasets. The comparison with other algorithms showed that the IKDC was superior to other algorithms in multiple clustering indicators.

Multiple Classifiers-Assisted Evolutionary Algorithm Based on Decomposition for High-Dimensional Multiobjective Problems

Surrogate-assisted multiobjective evolutionary algorithms (MOEAs) have advanced the field of computationally expensive optimization, but their progress is often restricted to low-dimensional problems. This manuscript presents a multiple classifiers-assisted evolutionary algorithm based on decomposition, which is adapted for high-dimensional expensive problems in terms of the following two insights. Compared to approximation-based surrogates, the accuracy of classification-based surrogates is robust for few high-dimensional training samples. Furthermore, multiple local classifiers can hedge the risk of overfitting issues. Accordingly, the proposed algorithm builds multiple classifiers with support vector machines (SVMs) on a decomposition-based multiobjective algorithm, wherein each local classifier is trained for a corresponding scalarization function. Experimental results confirm that the proposed algorithm is competitive to the state-of-the-art algorithms and computationally efficient as well.

Properties of eigenvalues and eigenvectors of large-dimensional sample correlation matrices

This paper is to study the properties of eigenvalues and eigenvectors of high-dimensional sample correlation matrices. We first improve the result of Jiang (Sankhyā 66 (2004) 35–48), Xiao and Zhou (J. Theoret. Probab. 23 (2010) 1–20) and the Theorem 1 of El Karoui (Ann. Appl. Probab. 19 (2009) 2362–2405), both concerning the limiting spectral distribution and the extreme eigenvalues of sample correlation matrices, by allowing a more general fourth moment condition. Then, we establish a central limit theorem (CLT) for the linear statistics of the eigenvectors of large sample correlation matrices. We discover that the difference between the functional CLT of the sample covariance matrix and the sample correlation matrix is fundamentally influenced by the direction of a nonrandom projection vector. In the special case where the square root of the correlation matrix is identity, the difference will be determined by the sum of the fourth powers of the entries of the projection vector. These results also indicate that the eigenmatrix of sample correlation matrices is not asymptotically Haar if the underlying distribution is Gaussian. In other words, the normalization based on the sample variances affects the asymptotic properties of the eigenmatrix of the Wishart matrix. Furthermore, we establish a theorem concerning CLT for the linear statistics of the eigenvectors of large sample covariance matrices. This theorem improves the main results in Bai, Miao and Pan (Ann. Probab. 35 (2007) 1532–1572), which requires the assumption that the fourth moment of the underlying variable matches the one of Gaussian distribution, as well as Theorem 1.3 in Pan and Zhou (Ann. Appl. Probab. 18 (2008) 1232–1270), which relaxed the Gaussian like fourth moment requirement but assumes the maximum entries of the projection vector converge to 0 uniformly. We illustrate the usefulness of the theoretical results through an application in communications.

Multiple discriminant preserving support subspace RBFNNs with graph similarity learning

In a high-dimensional sample space, the distribution characteristics of samples are complex. It is difficult to accurately describe the local distribution characteristics of samples directly in the original sample space. This paper proposes a multiple discriminant preserving support subspace radial basis function neural networks with graph similarity learning (DPSS-RBFNN) to describe the distribution characteristics of samples in the original high-dimensional space by multiple low-dimensional and simple discriminant preserving subspaces. DPSS-RBFNN includes the discriminant preserving support subspace (DPSS) learning module and the subspace distribution feature extraction (SDFE) module. In the DPSS learning module, the discriminativeness of each attribute and the joint discriminant between attributes are first considered to construct multiple subspaces. The discriminativeness of the features in these subspaces is not lower than that of the original samples. The graph similarity learning is used to measure the similarity of sample distributions between subspaces with different dimensions. Then multiple DPSSs are obtained. In SDFE module, the distribution characteristics of the samples are described by combining the local responses of the feature space extracted by the sub-RBFNN in each DPSS. The experimental results show that the proposed DPSS-RBFNN with few kernels can achieve higher accuracy in the recognition task than state-of-the-art algorithms.

ℓ 2 -Relaxation: With Applications to Forecast Combination and Portfolio Analysis

Abstract We propose ℓ2-relaxation, which is a novel convex optimization problem, to tackle forecast combination with many forecasts or minimum variance portfolio with many assets. ℓ2-relaxation minimizes the squared Euclidean norm of the weight vector subject to a set of relaxed linear inequalities to balance the bias and variance. It delivers optimality with approximately equal within-group weights when latent block equicorrelation patterns dominate the high dimensional sample variance-covariance matrix of the individual forecast errors or the assets. Its wide applicability is highlighted in three real data examples in microeconomics, macroeconomics, and finance.

Instance-wise multi-view representation learning

Multi-view representation learning aims to integrate multiple data information from different views to improve the task performance. The information contained in multi-view data is usually complex. Not only do different views contain different information, but also different samples of the same view contain different information. In the multi-view representation learning, most existing methods either simply treat each view/sample with equal importance, or set fixed or dynamic weights for different views/samples, which is not accurate enough to capture the information of dimensions of each sample and causes information redundancy, especially for high-dimensional samples. In this paper, we propose a novel unsupervised multi-view representation learning method based on instance-wise feature selection. A main advantage of instance-wise feature selection in this paper is that one can dynamically select dimensions that favor both view-specific representation learning and view-shared representation learning for each sample, thereby improving the performance from the perspective of model input. The proposed method consists of selector network, view-specific network and view-shared network. Specifically, selector network is used to obtain the selection template, which selects different number of dimensions conducive to representation learning from different samples to solve the sample heterogeneity problem; the view-specific network and view-shared network are used to extract the view-specific and view-shared representations, respectively. The selector network, view-shared network, and view-specific network are optimized alternately. Extensive experiments on various multi-view datasets with clustering and multi-label classification tasks demonstrate that the proposed method outperforms the state-of-the-art multi-view learning methods.

Reweighted Manifold Learning of Collective Variables from Enhanced Sampling Simulations.

Enhanced sampling methods are indispensable in computational chemistry and physics, where atomistic simulations cannot exhaustively sample the high-dimensional configuration space of dynamical systems due to the sampling problem. A class of such enhanced sampling methods works by identifying a few slow degrees of freedom, termed collective variables (CVs), and enhancing the sampling along these CVs. Selecting CVs to analyze and drive the sampling is not trivial and often relies on chemical intuition. Despite routinely circumventing this issue using manifold learning to estimate CVs directly from standard simulations, such methods cannot provide mappings to a low-dimensional manifold from enhanced sampling simulations, as the geometry and density of the learned manifold are biased. Here, we address this crucial issue and provide a general reweighting framework based on anisotropic diffusion maps for manifold learning that takes into account that the learning data set is sampled from a biased probability distribution. We consider manifold learning methods based on constructing a Markov chain describing transition probabilities between high-dimensional samples. We show that our framework reverts the biasing effect, yielding CVs that correctly describe the equilibrium density. This advancement enables the construction of low-dimensional CVs using manifold learning directly from the data generated by enhanced sampling simulations. We call our framework reweighted manifold learning. We show that it can be used in many manifold learning techniques on data from both standard and enhanced sampling simulations.

Testing Multivariate Normality Based on t-Representative Points

Testing multivariate normality is an ever-lasting interest in the goodness-of-fit area since the classical Pearson’s chi-squared test. Among the numerous approaches in the construction of tests for multivariate normality, normal characterization is one of the common approaches, which can be divided into the necessary and sufficient characterization and necessary-only characterization. We construct a test for multivariate normality by combining the necessary-only characterization and the idea of statistical representative points in this paper. The main idea is to transform a high-dimensional sample into a one-dimensional one through the necessary normal characterization and then employ the representative-point-based Pearson’s chi-squared test. A limited Monte Carlo study shows a considerable power improvement of the representative-point-based chi-square test over the traditional one. An illustrative example is given to show the supplemental function of the new test when used together with existing ones in the literature.