High-dimensional Samples Research Articles

A novel deterministic sampling approach for the reliability analysis of high-dimensional structures

Overcoming the “curse of dimensionality” in high-dimensional reliability analysis is still an enduring challenge. This paper proposes an innovative deterministic sampling method designed to overcome this challenge. The approach starts with a two-dimensional uniform point set, generated using the good lattice point method. This set is then refined through the cutting method to produce a specific number of points. A novel generating vector is computed based on this method, enabling the generation of the targeted high-dimensional point set through a strategic dimension-by-dimension mapping. Notably, this method eliminates the need for complex congruence computation and primitive root optimization, enhancing its efficiency for high-dimensional sampling. The resulting point set is deterministic and uniform, greatly reducing variability in reliability analysis. Then, the proposed approach is integrated into the fractional exponential moment-based maximum entropy method with the Box–Cox transform. This integration efficiently recovers the probability distribution for the limit state function (LSF) with high-dimensional inputs, enabling precise assessment of the failure probability. The efficacy of the proposed method is demonstrated through three high-dimensional numerical examples, involving both explicit and implicit LSFs, highlighting its applicability for high-dimensional reliability analysis of structures.

Multi-Frequency Microwave Sensing System with Frequency Selection Method for Pulverized Coal Concentration.

The accurate measurement of pulverized coal concentration (PCC) is crucial for optimizing the production efficiency and safety of coal-fired power plants. Traditional microwave attenuation methods typically rely on a single frequency for analysis while neglecting valuable information in the frequency domain, making them susceptible to the varying sensitivity of the signal at different frequencies. To address this issue, we proposed an innovative frequency selection method based on principal component analysis (PCA) and orthogonal matching pursuit (OMP) algorithms and implemented a multi-frequency microwave sensing system for PCC measurement. This method transcended the constraints of single-frequency analysis by employing a developed hardware system to control multiple working frequencies and signal paths. It measured insertion loss data across the sensor cross-section at various frequencies and utilized PCA to reduce the dimensionality of high-dimensional full-path insertion loss data. Subsequently, the OMP algorithm was applied to select the optimal frequency signal combination based on the contribution rates of the eigenvectors, enhancing the measurement accuracy through multi-dimensional fusion. The experimental results demonstrated that the multi-frequency microwave sensing system effectively extracted features from the high-dimensional PCC samples and selected the optimal frequency combination. Filed experiments conducted on five coal mills showed that, within a common PCC range of 0-0.5 kg/kg, the system achieved a minimum mean absolute error (MAE) of 1.41% and a correlation coefficient of 0.85. These results indicate that the system could quantitatively predict PCC and promptly detect PCC fluctuations, highlighting its immediacy and reliability.

Robust Estimation for Number of Factors in High Dimensional Factor Modeling via Spearman Correlation Matrix

Determining the number of factors in high-dimensional factor modeling is essential but challenging, especially when the data are heavy-tailed. In this article, we introduce a new estimator based on the spectral properties of Spearman sample correlation matrix under the high-dimensional setting, where both dimension and sample size tend to infinity proportionally. Our estimator is robust against heavy tails in either the common factors or idiosyncratic errors. The consistency of our estimator is established under mild conditions. Numerical experiments demonstrate the superiority of our estimator compared to existing methods. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Covariate balancing for high-dimensional samples in controlled experiments

In controlled experiments, achieving covariate balancing across all groups is crucial as it ensures that the estimated treatment effects are not confounded by the effects of covariates. This study proposes a mixed-integer nonlinear programming model to address the covariate balancing problem. Specifically, we introduce a new covariate imbalance measure, which is the maximum discrepancy in both the first and second central moments between any two groups. The second central moment can effectively capture the correlation of covariates in a physical sense, which is crucial for partitioning high-dimensional samples. A mixed-integer nonlinear programming model is constructed to minimize the proposed measure to obtain the optimal partitioning results. The nonlinear model is then linearized to accelerate the optimization process. We conduct computational experiments based on simulated datasets, including one-dimensional, two-dimensional, and three-dimensional Gaussian distributed samples, and a real clinic trial dataset. Compared to the conventional discrepancy-based method, our method achieves a 54.81% and a 40.6% reduction in the maximum discrepancy of partitioning results in the two-dimensional simulated Gaussian samples and the real clinic trial dataset, respectively. These results demonstrate the superiority of the proposed model in partitioning high-dimensional samples with correlated covariates compared with the conventional discrepancy-based method.

Large sample correlation matrices with unbounded spectrum

In this paper, we demonstrate that the diagonal of a high-dimensional sample covariance matrix stemming from n independent observations of a p-dimensional time series with finite fourth moments can be approximated in spectral norm by the diagonal of the population covariance matrix regardless of the spectral norm of the population covariance matrix. Our assumptions involve p and n tending to infinity, with p/n tending to a constant which might be positive or zero. Consequently, we investigate the asymptotic properties of the sample correlation matrix with a divergent spectrum, and we explore its applications by deriving the limiting spectral distribution for its eigenvalues and analyzing the convergence of divergent and non-divergent spiked eigenvalues under a generalized spiked correlation framework.

HiddenSinger: High-quality singing voice synthesis via neural audio codec and latent diffusion models

Recently, denoising diffusion models have demonstrated remarkable performance among generative models in various domains. However, in the speech domain, there are limitations in complexity and controllability to apply diffusion models for time-varying audio synthesis. Particularly, a singing voice synthesis (SVS) task, which has begun to emerge as a practical application in the game and entertainment industries, requires high-dimensional samples with long-term acoustic features. To alleviate the challenges posed by model complexity in the SVS task, we propose HiddenSinger, a high-quality SVS system using a neural audio codec and latent diffusion models. To ensure high-fidelity audio, we introduce an audio autoencoder that can encode audio into an audio codec as a compressed representation and reconstruct the high-fidelity audio from the low-dimensional compressed latent vector. Subsequently, we use the latent diffusion models to sample a latent representation from a musical score. In addition, our proposed model is extended to an unsupervised singing voice learning framework, HiddenSinger-U, to train the model using an unlabeled singing voice dataset. Experimental results demonstrate that our model outperforms previous models regarding audio quality. Furthermore, the HiddenSinger-U can synthesize high-quality singing voices of speakers trained solely on unlabeled data.

Label-Only Membership Inference Attack Based on Model Explanation

It is well known that machine learning models (e.g., image recognition) can unintentionally leak information about the training set. Conventional membership inference relies on posterior vectors, and this task becomes extremely difficult when the posterior is masked. However, current label-only membership inference attacks require a large number of queries during the generation of adversarial samples, and thus incorrect inference generates a large number of invalid queries. Therefore, we introduce a label-only membership inference attack based on model explanations. It can transform a label-only attack into a traditional membership inference attack by observing neighborhood consistency and perform fine-grained membership inference for vulnerable samples. We use feature attribution to simplify the high-dimensional neighborhood sampling process, quickly identify decision boundaries and recover a posteriori vectors. It also compares different privacy risks faced by different samples through finding vulnerable samples. The method is validated on CIFAR-10, CIFAR-100 and MNIST datasets. The results show that membership attributes can be identified even using a simple sampling method. Furthermore, vulnerable samples expose the model to greater privacy risks.

AI-based tree modeling for multi-point dioxin concentrations in municipal solid waste incineration

Numerous investigations have shown that the municipal solid waste incineration (MSWI) has become one of the major sources of dioxin (DXN) emissions. Currently, the primary issue that needs to be addressed for DXN emission reduction control is the online measurement of DXN. Data-driven AI algorithms enable real-time DXN concentration measurement, facilitating its control. However, researchers mainly focus on building models for DXN emissions at the stack. This approach does not allow for the construction of models that online measurement of DXN generation and absorption throughout the whole process. To achieve optimal pollution control, models that encompass the whole process are necessary, not just models focused on the stack. Therefore, this article focuses on modeling the whole process of DXN concentrations, including generation, adsorption, and emission. It uses machine learning techniques based on advanced tree-based data-driven deep and broad learning algorithms. The determination of data characteristics at different phases is grounded in the understanding of the DXN mechanism, offering a novel framework for DXN modeling. State-of-the-art tree-based models, including adaptive deep forest regression algorithm based on cross layer full connection, tree broad learning system, fuzzy forest regression, and aid modeling technologies, are applied to handle diverse data characteristics. These characteristics encompass high-dimensional small samples, low-dimensional ultra-small size samples, and medium-dimensional small samples across different phases related to DXN. The most interesting is the robust validation where the proposed a whole process tree-based model for DXN is validated using nearly one year of authentic data on DXN generation, adsorption, and emission phases in an MSWI plant of Beijing. The proposed modeling framework can be used to explore the mechanism characterization and support the pollution reduction optimal control.

TabDEG: Classifying differentially expressed genes from RNA-seq data based on feature extraction and deep learning framework.

Traditional differential expression genes (DEGs) identification models have limitations in small sample size datasets because they require meeting distribution assumptions, otherwise resulting high false positive/negative rates due to sample variation. In contrast, tabular data model based on deep learning (DL) frameworks do not need to consider the data distribution types and sample variation. However, applying DL to RNA-Seq data is still a challenge due to the lack of proper labeling and the small sample size compared to the number of genes. Data augmentation (DA) extracts data features using different methods and procedures, which can significantly increase complementary pseudo-values from limited data without significant additional cost. Based on this, we combine DA and DL framework-based tabular data model, propose a model TabDEG, to predict DEGs and their up-regulation/down-regulation directions from gene expression data obtained from the Cancer Genome Atlas database. Compared to five counterpart methods, TabDEG has high sensitivity and low misclassification rates. Experiment shows that TabDEG is robust and effective in enhancing data features to facilitate classification of high-dimensional small sample size datasets and validates that TabDEG-predicted DEGs are mapped to important gene ontology terms and pathways associated with cancer.

Two-sample test for high-dimensional covariance matrices: A normal-reference approach

Testing the equality of the covariance matrices of two high-dimensional samples is a fundamental inference problem in statistics. Several tests have been proposed but they are either too liberal or too conservative when the required assumptions are not satisfied which attests that they are not always applicable in real data analysis. To overcome this difficulty, a normal-reference test is proposed and studied in this paper. It is shown that under some regularity conditions and the null hypothesis, the proposed test statistic and a chi-squared-type mixture have the same limiting distribution. It is then justified to approximate the null distribution of the proposed test statistic using that of the chi-squared-type mixture. The distribution of the chi-squared-type mixture can be well approximated using a three-cumulant matched chi-squared-approximation with its approximation parameters consistently estimated from the data. The asymptotic power of the proposed test under a local alternative is also established. Simulation studies and a real data example demonstrate that the proposed test works well in general scenarios and outperforms the existing competitors substantially in terms of size control.

CPMI: comprehensive neighborhood-based perturbed mutual information for identifying critical states of complex biological processes

BackgroundThere exists a critical transition or tipping point during the complex biological process. Such critical transition is usually accompanied by the catastrophic consequences. Therefore, hunting for the tipping point or critical state is of significant importance to prevent or delay the occurrence of catastrophic consequences. However, predicting critical state based on the high-dimensional small sample data is a difficult problem, especially for single-cell expression data.ResultsIn this study, we propose the comprehensive neighbourhood-based perturbed mutual information (CPMI) method to detect the critical states of complex biological processes. The CPMI method takes into account the relationship between genes and neighbours, so as to reduce the noise and enhance the robustness. This method is applied to a simulated dataset and six real datasets, including an influenza dataset, two single-cell expression datasets and three bulk datasets. The method can not only successfully detect the tipping points, but also identify their dynamic network biomarkers (DNBs). In addition, the discovery of transcription factors (TFs) which can regulate DNB genes and nondifferential ‘dark genes’ validates the effectiveness of our method. The numerical simulation verifies that the CPMI method is robust under different noise strengths and is superior to the existing methods on identifying the critical states.ConclusionsIn conclusion, we propose a robust computational method, i.e., CPMI, which is applicable in both the bulk and single cell datasets. The CPMI method holds great potential in providing the early warning signals for complex biological processes and enabling early disease diagnosis.

Fisher discrimination multiple kernel dictionary learning for robust identification of nonlinear features in machinery health monitoring

In sparse representation-based classification, Fisher discrimination dictionary learning (FDDL) has attracted widespread attention due to its advantages such as fewer parameters and good dictionary discriminability. However, due to its linear nature, it is difficult to handle nonlinear features. Kernel-based nonlinear transformation enables dictionary learning to capture nonlinear features embedded in samples, but traditional single-kernel methods have limited performance. In this paper, a Fisher discrimination multiple kernel dictionary learning (FDMKDL) method is proposed, in which Fisher discriminative criterion is imposed on both high-dimensional samples and coding coefficients. Specifically, we derive the kernel version of FDDL, i.e., Fisher discrimination kernel dictionary learning (FDKDL) to learn nonlinear features and promote dictionary discriminability. Meanwhile, to avoid the problem of poor adaptability and possible manual selection caused by single-kernel methods, we further derive a flexible multiple kernel learning (MKL) framework, which utilizes the complementary information of multiple kernel functions to adaptively obtain the optimal weights and synthesize a discriminative kernel space. Finally, FDMKDL combines FDKDL with this MKL framework to obtain a more discriminative dictionary. Experimental evaluations conducted on two datasets demonstrate the effectiveness and robustness of the proposed FDMKDL method in learning nonlinear features for machinery health monitoring when compared to several state-of-the-art methods.

TSWIFT, a novel method for iterative staining of embedded and mounted human brain sections

Comprehensive characterization of protein networks in mounted brain tissue represents a major challenge in brain and neurodegenerative disease research. In this study, we develop a simple staining method, called TSWIFT, to iteratively stain pre-mounted formalin fixed, paraffin embedded (FFPE) brain sections, thus enabling high-dimensional sample phenotyping. We show that TSWIFT conserves tissue architecture and allows for relabeling a single mounted FFPE sample more than 10 times, even after prolonged storage at 4 °C. Our results establish TSWIFT as an efficient method to obtain integrated high-dimensional knowledge of cellular proteomes by analyzing mounted FFPE human brain tissue.

Joint Anchor Graph Embedding and Discrete Feature Scoring for Unsupervised Feature Selection.

The success of existing unsupervised feature selection (UFS) methods heavily relies on the assumption that the intrinsic relationships among original high-dimensional (HD) data samples exist in the discriminative low-dimension (LD) subspace. However, previous UFS methods commonly construct pairwise graphs and employ l2,1 -norm regularization to severally preserve the local structure and calculate the score of features, which is computationally complex and easy to get stuck into local optimum, so that those approaches cannot be applied in dealing with large-scale datasets in practice. To overcome this challenge, we propose a novel UFS method, in which a novel anchor graph embedding paradigm is designed to extract the local neighborhood relationships among data samples by reducing the computational complexity of graph construction to be linear in the number of data. Moreover, to improve the optimality of selected features as well as the performance of downstream tasks, we propose a discrete feature scoring mechanism, which imposes orthogonal l2,0 -norm constraints on learned projections, in order to enhance the distinction of feature scores as well as reduce the probability of falling into local optimum. In addition, solving the proposed nonconvex and nonsmooth NP-hard problem is challenging, and we present an efficient optimization algorithm to address it and acquire a closed-form solution of the transformation matrix. Extensive experiments demonstrate the effectiveness and efficiency of the proposed UFS by comparison with several state-of-the-art approaches to clustering and image segmentation tasks.

Robust Spherical Laplacian Embedding.

Laplacian embedding (LE) aims to project high-dimensional input data samples, which often contain nonlinear structures, into a low-dimensional space. However, existing distance functions used in the embedding space fail to provide discriminative representations for real-world datasets, especially those related to text analysis or image processing. Cosine similarity measurements are advantageous in dealing with sparse data but are fragile to the impact of outliers and noise from samples or data features. In this work, we propose robust spherical LE (RS-LE), a powerful method that builds the LE on a novel metric that unifies the Euclidean distance and cosine similarity in a spherical space using a robust lp,p -norm. We introduce an efficient iterative algorithm, the proximal alternating linearized minimization (ALM) method, to solve the difficult optimization problem that arises from the new metric, which is neither convex nor smooth. This algorithm allows the solution to achieve both global and objective convergence. Our findings are presented in rigorous theoretical analysis and validated in experiments.

Interpretable metric learning in comparative metagenomics: The adaptive Haar-like distance.

Random forests have emerged as a promising tool in comparative metagenomics because they can predict environmental characteristics based on microbial composition in datasets where β-diversity metrics fall short of revealing meaningful relationships between samples. Nevertheless, despite this efficacy, they lack biological insight in tandem with their predictions, potentially hindering scientific advancement. To overcome this limitation, we leverage a geometric characterization of random forests to introduce a data-driven phylogenetic β-diversity metric, the adaptive Haar-like distance. This new metric assigns a weight to each internal node (i.e., split or bifurcation) of a reference phylogeny, indicating the relative importance of that node in discerning environmental samples based on their microbial composition. Alongside this, a weighted nearest-neighbors classifier, constructed using the adaptive metric, can be used as a proxy for the random forest while maintaining accuracy on par with that of the original forest and another state-of-the-art classifier, CoDaCoRe. As shown in datasets from diverse microbial environments, however, the new metric and classifier significantly enhance the biological interpretability and visualization of high-dimensional metagenomic samples.

Asymptotic behavior of some multicategory classification methods for high-dimensional data

We consider multicategory extensions of binary discrimination methods via one-versus-one (OVO) or one-versus-rest (OVR) methodologies, focusing on extensions of the binary classification by linear mean difference (MD), support vector machine (SVM), maximal data piling (MDP), and distance weighted discrimination (DWD) via OVO, and the multicategory extension of MD via OVR, in the context of high-dimensional and low sample size (HDLSS) data. The asymptotic behavior of OVO-MD, OVO-SVM, OVO-MDP and OVO-DWD is described when the dimension of the data increases and the sample size is fixed, in terms of the probabilities of correct classification of a new data point, finding sufficient conditions for the correct classification probabilities to converge to one as the dimension approaches infinity. As in the binary case, OVO-MD, OVO-SVM and OVO-MDP have the same asymptotic behavior while OVO-DWD could behave differently. We also consider the asymptotic behavior of the OVR-MD methodology providing necessary and sufficient conditions for a new data point of a given class to be correctly classified with probability tending to one. A simulation experiment is conducted to further compare the methodologies, and consider the four binary methods in the OVR case. We evaluate the performance of the considered methods using a microarray data set.

High-dimensional Statistical Analysis and Its Application to an ALMA Map of NGC 253

In astronomy, if we denote the dimension of data as d and the number of samples as n, we often find a case with n ≪ d. Traditionally, such a situation is regarded as ill-posed, and there was no choice but to discard most of the information in data dimensions to let d < n. The data with n ≪ d is referred to as the high-dimensional low sample size (HDLSS). To deal with HDLSS problems, a method called high-dimensional statistics has rapidly developed in the last decade. In this work, we first introduce high-dimensional statistical analysis to the astronomical community. We apply two representative methods in the high-dimensional statistical analysis methods, noise-reduction principal component analysis (NRPCA) and automatic sparse principal component analysis (A-SPCA), to a spectroscopic map of a nearby archetype starburst galaxy NGC 253 taken by the Atacama Large Millimeter/submillimeter Array (ALMA). The ALMA map is an example of a typical HDLSS data set. First, we analyzed the original data, including the Doppler shift due to the systemic rotation. High-dimensional PCA can precisely describe the spatial structure of the rotation. We then applied to the Doppler-shift corrected data to analyze more subtle spectral features. NRPCA and R-SPCA were able to quantify the very complicated characteristics of the ALMA spectra. Particularly, we were able to extract information on the global outflow from the center of NGC 253. This method can also be applied not only to spectroscopic survey data, but also to any type of data with a small sample size and large dimension.

A hybrid feature selection algorithm combining information gain and grouping particle swarm optimization for cancer diagnosis.

Cancer diagnosis based on machine learning has become a popular application direction. Support vector machine (SVM), as a classical machine learning algorithm, has been widely used in cancer diagnosis because of its advantages in high-dimensional and small sample data. However, due to the high-dimensional feature space and high feature redundancy of gene expression data, SVM faces the problem of poor classification effect when dealing with such data. Based on this, this paper proposes a hybrid feature selection algorithm combining information gain and grouping particle swarm optimization (IG-GPSO). The algorithm firstly calculates the information gain values of the features and ranks them in descending order according to the value. Then, ranked features are grouped according to the information index, so that the features in the group are close, and the features outside the group are sparse. Finally, grouped features are searched using grouping PSO and evaluated according to in-group and out-group. Experimental results show that the average accuracy (ACC) of the SVM on the feature subset selected by the IG-GPSO is 98.50%, which is significantly better than the traditional feature selection algorithm. Compared with KNN, the classification effect of the feature subset selected by the IG-GPSO is still optimal. In addition, the results of multiple comparison tests show that the feature selection effect of the IG-GPSO is significantly better than that of traditional feature selection algorithms. The feature subset selected by IG-GPSO not only has the best classification effect, but also has the least feature scale (FS). More importantly, the IG-GPSO significantly improves the ACC of SVM in cancer diagnostic.

Data-Efficient Generation of Protein Conformational Ensembles with Backbone-to-Side-Chain Transformers.

Excitement at the prospect of using data-driven generative models to sample configurational ensembles of biomolecular systems stems from the extraordinary success of these models on a diverse set of high-dimensional sampling tasks. Unlike image generation or even the closely related problem of protein structure prediction, there are currently no data sources with sufficient breadth to parametrize generative models for conformational ensembles. To enable discovery, a fundamentally different approach to building generative models is required: models should be able to propose rare, albeit physical, conformations that may not arise in even the largest data sets. Here we introduce a modular strategy to generate conformations based on "backmapping" from a fixed protein backbone that (1) maintains conformational diversity of the side chains and (2) couples the side-chain fluctuations using global information about the protein conformation. Our model combines simple statistical models of side-chain conformations based on rotamer libraries with the now ubiquitous transformer architecture to sample with atomistic accuracy. Together, these ingredients provide a strategy for rapid data acquisition and hence a crucial ingredient for scalable physical simulation with generative neural networks.