High-dimensional Input Research Articles

Modeling Functional Brain Networks for ADHD via Spatial Preservation-Based Neural Architecture Search.

Modeling functional brain networks (FBNs) for attention deficit hyperactivity disorder (ADHD) has sparked significant interest since the abnormal functional connectivity is discovered in certain functional magnetic resonance imaging (fMRI)-based brain regions compared to typical developmental control (TC) individuals. However, existing models for modeling FBNs generally use dimensionality reduction techniques to process the high dimensional input data, which results in confusion and an inaccurate representation of voxel interactions between spatially close brain regions, causing misdiagnosis of the disease. To address these issues, we propose a spatial preservation-based neural architecture search (SP-NAS) for FBNs modeling in ADHD. The main work includes three-fold: 1) A spatial preservation module is designed to embed original spatial information into dimensionality reduction data, addressing the challenge of a large number of parameters in the original data and mitigating disease misdiagnosis resulting from voxel confusion between different brain regions caused by dimensionality reduction. 2) A search space using more suitable search operations is constructed to efficiently extract spatial-temporal interaction characteristics of fMRI data in ADHD while narrowing the search space. 3) Cross-regional association differences between ADHD and TC groups are explored for ADHD auxiliary diagnosis since the abnormal activation regions of ADHD relative to TC on the brain regions and the abnormal connectivity between the lesion brain regions are identified. Model validation results on the ADHD-200 dataset show that the FBNs obtained from SP-NAS not only achieve competitive results in ADHD diagnosis but also reveal abnormal connections in the lesion regions of ADHD consistent with clinical diagnosis.

A fusion algorithm of multidimensional element space mapping architecture for SOC estimation of lithium-ion batteries under dynamic operating conditions

Accurate estimation of the state of charge (SOC) is pivotal for ensuring the reliable operation of lithium-ion batteries in electric vehicles, particularly under complex real-world conditions. This paper introduces a novel SOC estimation method named Multidimensional Elemental Space Mapping Architecture (MESMA), which integrates the battery equivalent circuit model with a data-driven approach to tackle adaptability under intricate conditions and high-dimensional input data correlation. Initially, a least squares method incorporating a forgetting factor is employed to identify the battery circuit model parameters and analyze their correlation with SOC. Subsequently, a feature fusion algorithm is introduced to discern spatial variables highly correlated with SOC by leveraging both original voltage and current data and circuit model parameters as inputs to a Convolutional Neural Network (CNN) model, and an eight-input and one-output spatially mapped feature CNN model is established. Finally, the XGBoost model is harnessed for battery SOC estimation. The effectiveness and adaptability of MESMA are validated using experimental data from both public single condition and mixed condition datasets, including FUDS, Mixed-1, etc. The results unequivocally demonstrate that MESMA accurately estimates the SOC of lithium-ion batteries under varying operating conditions. Furthermore, it exhibits remarkable adaptability and stability across different temperatures, effectively capturing SOC trends.

A deep autoencoder for electric double layer capacitance prediction in electrochemical sensors

This study explores the application of a deep autoencoder neural network to accurately predict the electric double layer capacitance from real-world parameters in binary, asymmetric electrolytes under low concentration conditions. By utilizing a modest simulation-based dataset of just 250 samples, the deep autoencoder neural network model developed in this study effectively predicted the capacitance by learning the critical features and relationships of the electric double layer model and encoding this learned representation into a low-dimensional latent space. From the latent variables, the decoder block of the neural network learned to effectively recreate the high-dimensional input. To enhance the model's robustness, prevent overfitting, and better simulate real-world conditions, noise was incorporated into the training and test data. The model demonstrated strong performance across various conditions, such as ionic size, ionic charge, and surface potential, yielding satisfactory results on both clean and noisy test datasets. A key feature of this approach was the mapping of real-world electric double layer parameters to the latent variables of the model, allowing for direct input of physical parameters to predict the electric double layer capacitance. This research highlights the potential of machine learning techniques to expedite the design and analysis of complex multi-physics systems such as electrochemical sensors by reducing the dependence on extensive domain expertise throughout the design process.

Multivariate sensitivity-adaptive polynomial chaos expansion for high-dimensional surrogate modeling and uncertainty quantification

This work develops a novel basis-adaptive method for constructing anisotropic polynomial chaos expansions of multidimensional (vector-valued, multi-output) model responses. The adaptive basis selection is based on multivariate sensitivity analysis metrics that can be estimated by post-processing the polynomial chaos expansion and results in a common anisotropic polynomial basis for the vector-valued response. This allows the application of the method to problems with up to moderately high-dimensional model inputs (in the order of tens) and up to very high-dimensional model responses (in the order of thousands). The method is applied to different engineering test cases for surrogate modeling and uncertainty quantification, including use cases related to electric machine and power grid modeling and simulation, and is found to produce highly accurate results with comparatively low data and computational demand.

Supervised kernel principal component analysis-polynomial chaos-Kriging for high-dimensional surrogate modelling and optimization

Surrogate-based optimization (SBO) approach is becoming more and more popular in the expensive aerodynamic design of aircraft. However, with increasing number of design variables required for parameterizing a complex shape, SBO is suffering from the serious difficulty of the curse of dimensionality. To ameliorate this issue, a supervised nonlinear dimensionality-reduction surrogate modelling method was proposed. Such a method combines the supervised kernel principal component analysis (SKPCA) and polynomial chaos-Kriging (PCK) techniques into the jointly surrogate modelling process and adaptively establishes the accurate mapping from the high-dimensional inputs to the output of the system. This SKPCA-PCK method, which fully considers the effect of inputs on outputs and adaptively trains these hyper-parameters in the surrogate modelling process, escapes from the low prediction accuracy and instability of the surrogate model in conjunction with current linear or unsupervised dimensionality-reduction methods. Further, an efficient SKPCA-PCK-based global optimization method for high-dimensional aerodynamic design was developed. The performance of the proposed method is examined by investigating two numerical examples, the transonic RAE2822 airfoil and the wing of the NASA Common Research Model. Results demonstrate that the proposed SKPCA-PCK method significantly improves the modelling efficiency and accuracy compared to the unsupervised linear PCA-Kriging method. More importantly, the proposed SKPCA-PCK-based optimization method provides better performance and an appreciably higher optimization efficiency for expensive single-point and robust aerodynamic design involving high-dimensional design variables compared to the Kriging-based optimization method. These results provide further evidence that the proposed method provides a promising approach for mitigating the curse of dimensionality in SBO.

A Sequential Stein's Method for Faster Training of Additive Index Models.

The additive index models (AIMs) can be viewed as a kind of artificial neural networks based on nonparametric activation or so-called ridge functions. Recently, they are shown to achieve enhanced explainability after incorporating various interpretability constraints. However, the training of AIMs by either the backfitting algorithm or the joint stochastic optimization is known to be very slow for especially high dimensional inputs. In this article, we propose a novel sequential approach based on the celebrated Stein's lemma. The proposed SeqStein method can successfully decouple the training of AIMs into two separable steps, namely, the following: 1) Stein's estimation of the projection indices and 2) nonparametric estimation of ridge functions using the smoothing splines. We show through numerical experiments that the SeqStein algorithm is not only more efficient for training AIMs, but also inclined to produce more interpretable models that have smooth ridge functions with sparse and nearly orthogonal projection indices.

Predictive modeling using Copula Particle Filter and Adaptive Network-Based Fuzzy Inference

This paper introduces a novel prediction algorithm, CPF-ANFIS, designed to overcome the challenges posed by high-dimensional input data in Adaptive Neuro-Fuzzy Inference Systems (ANFIS). ANFIS's performance deteriorates with increasing input dimensionality due to the distortion of its membership functions. To address this limitation, CPF-ANFIS leverages a two-stage approach: A Copula Particle Filter (CPF) for robust state estimation and ANFIS for nonlinear mapping. By incorporating copulas, CPF effectively addresses the impoverishment and degeneracy problems commonly encountered in traditional particle filters. This enhanced robustness allows for more accurate state estimation, which in turn improves the overall performance of the CPF-ANFIS algorithm. By decoupling state estimation from nonlinear modeling, CPF-ANFIS effectively mitigates the curse of dimensionality. The proposed method is evaluated on real-world applications, such as hybrid PV-wind systems and SLAM. Experimental results demonstrate that CPF-ANFIS consistently outperforms ANFIS and the Copula Particle Filter individually, as well as previously proposed methods such as ANFIS-PF, highlighting its effectiveness in achieving accurate predictions under challenging conditions. The results show that the CPF-ANFIS algorithm increases prediction accuracy by at least 5% compared to using each algorithm separately.

Synergizing Quality-Diversity with Descriptor-Conditioned Reinforcement Learning

A hallmark of intelligence is the ability to exhibit a wide range of effective behaviors. Inspired by this principle, Quality-Diversity algorithms, such as, are evolutionary methods designed to generate a set of diverse and high-fitness solutions. However, as a genetic algorithm, relies on random mutations, which can become inefficient in high-dimensional search spaces, thus limiting its scalability to more complex domains, such as learning to control agents directly from high-dimensional inputs. To address this limitation, advanced methods like and have been developed, which combine actor-critic techniques from Reinforcement Learning with, significantly enhancing the performance and efficiency of Quality-Diversity algorithms in complex, high-dimensional tasks. While these methods have successfully leveraged the trained critic to guide more effective mutations, the potential of the trained actor remains underutilized in improving both the quality and diversity of the evolved population. In this work, we introduce, an extension of that utilizes the descriptor-conditioned actor as a generative model to produce diverse solutions, which are then injected into the offspring batch at each generation. Additionally, we present an empirical analysis of the fitness and descriptor reproducibility of the solutions discovered by each algorithm. Finally, we present a second empirical analysis shedding light on the synergies between the different variations operators and explaining the performance improvement from to.

The prediction of bolted joint loosening state under transverse load based on a data-driven model

The impact of transverse load results in the loosening of bolted joints. The loosening condition is determined based on the transverse load and relevant parameters of the bolted joint. To efficiently forecast the loosening state of bolts under various transverse loads and relevant parameters, this study proposes a predictive model for the loosening state of bolted joints. Initially, the relationship between the loosening criteria and relevant parameters is established according to the bolt’s force conditions. Through elementary effect analysis, the high-dimensional representation model is improved, breaking down the loosening criteria function with high-dimensional parameter inputs into a series of low-dimensional functions. Subsequently, the prediction of the loosening state is achieved by training artificial neural networks to approximate multiple low-dimensional functions. The improved predictive model is utilized to determine the critical load for bolted joint loosening. Validation indicates the accuracy of the proposed model. Notably, the influence of the bearing friction coefficient and preload on the critical transverse load is more pronounced compared to the thread friction coefficient.

A nested non-intrusive stochastic isogeometric method for nonlinear thermal vibration of FGM plates under random loading

This paper introduces an adaptive nested non-invasive dynamic SIGA method based on RBFNN for the nonlinear thermal vibration analysis of FGM plates under random loading. This method is robust and accurate in high-dimensional input spaces regardless of the sample sequence, addresses the low convergence rate of QMCS, and solves the stochastic dynamic response probability density function under random loading. Firstly, the multidimensional input space for random loading is constructed based on several mutually independent random variables via the stationary Gaussian stochastic process simulation technique. The nonlinear FGM plate model subjected to random loading in the thermal environment is modeled using HSDT with von Kármán strain-displacement relation within the IGA framework. Subsequently, a nested non-invasive dynamic response surface analysis method, SIGA-RBFNN, is proposed based on the RBFNN technique. Analysis of the FGM plate response under random loading demonstrates that SIGA-RBFNN, compared to tensor-product-based SIGA methods, is less affected by sample sequence types. It effectively addresses high-dimensional stochasticity and offers significant advantages in robustness, accuracy, and efficiency. Finally, SIGA-RBFNN compensates for the low convergence speed of QMCS and successfully solves the FGM plate displacement response statistics and probability density function under random loading.

Reinforcement learning-based drone simulators: survey, practice, and challenge

Recently, machine learning has been very useful in solving diverse tasks with drones, such as autonomous navigation, visual surveillance, communication, disaster management, and agriculture. Among these machine learning, two representative paradigms have been widely utilized in such applications: supervised learning and reinforcement learning. Researchers prefer to use supervised learning, mostly based on convolutional neural networks, because of its robustness and ease of use but yet data labeling is laborious and time-consuming. On the other hand, when traditional reinforcement learning is combined with the deep neural network, it can be a very powerful tool to solve high-dimensional input problems such as image and video. Along with the fast development of reinforcement learning, many researchers utilize reinforcement learning in drone applications, and it often outperforms supervised learning. However, it usually requires the agent to explore the environment on a trial-and-error basis which is high cost and unrealistic in the real environment. Recent advances in simulated environments can allow an agent to learn by itself to overcome these drawbacks, although the gap between the real environment and the simulator has to be minimized in the end. In this sense, a realistic and reliable simulator is essential for reinforcement learning training. This paper investigates various drone simulators that work with diverse reinforcement learning architectures. The characteristics of the reinforcement learning-based drone simulators are analyzed and compared for the researchers who would like to employ them for their projects. Finally, we shed light on some challenges and potential directions for future drone simulators.

Seeing the primary tumor because of all the trees: Cancer type prediction on low-dimensional data.

The Cancer of Unknown Primary (CUP) syndrome is characterized by identifiable metastases while the primary tumor remains hidden. In recent years, various data-driven approaches have been suggested to predict the location of the primary tumor (LOP) in CUP patients promising improved diagnosis and outcome. These LOP prediction approaches use high-dimensional input data like images or genetic data. However, leveraging such data is challenging, resource-intensive and therefore a potential translational barrier. Instead of using high-dimensional data, we analyzed the LOP prediction performance of low-dimensional data from routine medical care. With our findings, we show that such low-dimensional routine clinical information suffices as input data for tree-based LOP prediction models. The best model reached a mean Accuracy of 94% and a mean Matthews correlation coefficient (MCC) score of 0.92 in 10-fold nested cross-validation (NCV) when distinguishing four types of cancer. When considering eight types of cancer, this model achieved a mean Accuracy of 85% and a mean MCC score of 0.81. This is comparable to the performance achieved by approaches using high-dimensional input data. Additionally, the distribution pattern of metastases appears to be important information in predicting the LOP.

Understanding human amygdala function with artificial neural networks.

The amygdala is a cluster of subcortical nuclei that receives diverse sensory inputs and projects to the cortex, midbrain and other subcortical structures. Numerous accounts of amygdalar contributions to social and emotional behavior have been offered, yet an overarching description of amygdala function remains elusive. Here we adopt a computationally explicit framework that aims to develop a model of amygdala function based on the types of sensory inputs it receives, rather than individual constructs such as threat, arousal, or valence. Characterizing human fMRI signal acquired as participants viewed a full-length film, we developed encoding models that predict both patterns of amygdala activity and self-reported valence evoked by naturalistic images. We use deep image synthesis to generate artificial stimuli that distinctly engage encoding models of amygdala subregions that systematically differ from one another in terms of their low-level visual properties. These findings characterize how the amygdala compresses high-dimensional sensory inputs into low-dimensional representations relevant for behavior.

Feature learning and generalization error analysis of two-layer linear neural networks for high-dimensional inputs

AbstractIt is well known that a model can generalize even when it completely interpolates the training data, which is known as the benign overfitting. Indeed, several work have theoretically revealed that the minimum-norm interpolator can exhibit the benign overfitting. On the other hand, deep learning models such as two-layer neural networks have been reported to outperform “shallow” learning models such as kernel methods under appropriate model sizes by adaptively learning the basis functions to the data. This mechanism is called feature learning, and it is known empirically to be beneficial even when the model size is large. However, it is generally difficult to show that benign overfitting occurs in learning models with feature learning especially for regression problems. In this study, we then analyze the predictive error of the estimator after one step feature learning in a two-layer linear neural network optimized by gradient descent methods and study the effect of feature learning on benign overfitting. The results show that feature learning reduces bias compared to a one-layer linear regression model without feature learning, especially when the eigenvalues of the covariance of input decay slowly. On the other hand, we clarify that the variance is hardly changed by feature learning. This differs significantly from the results for benign overfitting in the situation without feature learning and indicates the usefulness of feature learning.

Multifidelity Methodology for Reduced-Order Models with High-Dimensional Inputs

In the early stages of aerospace design, reduced-order models (ROMs) are crucial for minimizing computational costs associated with using physics-rich field information in many-query scenarios requiring multiple evaluations. The intricacy of aerospace design demands the use of high-dimensional design spaces to capture detailed features and design variability accurately. However, these spaces introduce significant challenges, including the curse of dimensionality, which stems from both high-dimensional inputs and outputs necessitating substantial training data and computational effort. To address these complexities, this study introduces a novel multifidelity, parametric, and nonintrusive ROM framework designed for high-dimensional contexts. It integrates machine learning techniques for manifold alignment and dimension reduction—employing proper orthogonal decomposition and model-based active subspace—with multifidelity regression for ROM construction. Our approach is validated through two test cases: the 2D RAE 2822 airfoil and the 3D NASA CRM wing, assessing various fidelity levels, training data ratios, and sample sizes. Compared to the single-fidelity principal component–active subspace (PCAS) method, our multifidelity solution offers improved cost-accuracy benefits and achieves better predictive accuracy with reduced computational demands. Moreover, our methodology outperforms the manifold-aligned ROM method by 50% in handling scenarios with large input dimensions, underscoring its efficacy in addressing the complex challenges of aerospace design.

Pre-frontal cortex guides dimension-reducing transformations in the occipito-ventral pathway for categorization behaviors

To interpret our surroundings, the brain uses a visual categorization process. Current theories and models suggest that this process comprises a hierarchy of different computations that transforms complex, high-dimensional inputs into lower-dimensional representations (i.e., manifolds) in support of multiple categorization behaviors. Here, we tested this hypothesis by analyzing these transformations reflected in dynamic MEG source activity while individual participants actively categorized the same stimuli according to different tasks: face expression, face gender, pedestrian gender, and vehicle type. Results reveal three transformation stages guided by the pre-frontal cortex. At stage 1 (high-dimensional, 50-120ms), occipital sources represent both task-relevant and task-irrelevant stimulus features; task-relevant features advance into higher ventral/dorsal regions, whereas task-irrelevant features halt at the occipital-temporal junction. At stage 2 (121-150ms), stimulus feature representations reduce to lower-dimensional manifolds, which then transform into the task-relevant features underlying categorization behavior over stage 3 (161-350ms). Our findings shed light on how the brain's network mechanisms transform high-dimensional inputs into specific feature manifolds that support multiple categorization behaviors.

Prognostic value of clinical and radiomic parameters in patients with liver metastases from uveal melanoma.

Approximately every second patient with uveal melanoma develops distant metastases, with the liver as the predominant target organ. While the median survival after diagnosis of distant metastases is limited to a year, yet-to-be-defined subgroups of patients experience a more favorable outcome. Therefore, prognostic biomarkers could help identify distinct risk groups to guide patient counseling, therapeutic decision-making, and stratification of study populations. To this end, we retrospectively analyzed a cohort of 101 patients with newly diagnosed hepatic metastases from uveal melanoma by using Cox-Lasso regression machine learning, adapted to a high-dimensional input parameter space. We show that substantial binary risk stratification can be performed, based on (i) clinical and laboratory parameters, (ii) measures of quantitative overall hepatic tumor burden, and (iii) radiomic parameters. Yet, combining two or all three domains failed to improve prognostic separation of patients. Additionally, we identified highly relevant clinical parameters (including lactate dehydrogenase, thrombocyte counts, aspartate transaminase, and the metastasis-free interval) at first diagnosis of metastatic disease as predictors for time-to-treatment failure and overall survival. Taken together, the risk stratification models, built by our machine-learning algorithm, identified a comparable and independent prognostic value of clinical, radiological, and radiomic parameters in uveal melanoma patients with hepatic metastases.

Active learning for adaptive surrogate model improvement in high-dimensional problems

This paper investigates a novel approach to efficiently construct and improve surrogate models in problems with high-dimensional input and output. In this approach, the principal components and corresponding features of the high-dimensional output are first identified. For each feature, the active subspace technique is used to identify a corresponding low-dimensional subspace of the input domain; then a surrogate model is built for each feature in its corresponding active subspace. A low-dimensional adaptive learning strategy is proposed to identify training samples to improve the surrogate model. In contrast to existing adaptive learning methods that focus on a scalar output or a small number of outputs, this paper addresses adaptive learning with high-dimensional input and output, with a novel learning function that balances exploration and exploitation, i.e., considering unexplored regions and high-error regions, respectively. The adaptive learning is in terms of the active variables in the low-dimensional space, and the newly added training samples can be easily mapped back to the original space for running the expensive physics model. The proposed method is demonstrated for the numerical simulation of an additive manufacturing part, with a high-dimensional field output quantity of interest (residual stress) in the component that has spatial variability due to the stochastic nature of multiple input variables (including process variables and material properties). Various factors in the adaptive learning process are investigated, including the number of training samples, range and distribution of the adaptive training samples, contributions of various errors, and the importance of exploration versus exploitation in the learning function.

A dimension reduction-based Kriging modeling method for high-dimensional time-variant uncertainty propagation and global sensitivity analysis

Surrogate models have been widely used in the uncertainty propagation and global sensitivity analysis of complex evaluation-expensive engineering problems. However, the construction of high-accuracy surrogate models for time-variant problems with high-dimensional inputs and outputs using a small number of training samples remains a challenge. To address this challenge, we propose a dimension reduction-based Kriging modeling (KMDR) method for high-dimensional time-variant uncertainty propagation and global sensitivity analysis. Singular value decomposition is performed on the original time-variant response to extract principal components from high-dimensional outputs. And the improved sufficient dimension reduction (ISDR) is performed on high-dimensional inputs to identify the latent input space with respect to each principal component of outputs. A ladle estimator with a rigorous mathematical definition is then employed to determine the number of principal components of the outputs and the dimensionalities of the latent input spaces. The ladle estimator considers variabilities in both eigenvalues and eigenvectors of a matrix and can determine the latent dimensionality more accurately and efficiently than existing approaches. Subsequently, Kriging models between high-dimensional inputs and each principal component of the outputs are constructed based on a newly devised Kriging kernel, which embeds the information of the ISDR into the kernel function and can achieve higher accuracy than directly constructing Kriging models between latent inputs and outputs. In addition, a generalized variance-based sensitivity index, which can quantify the effects of the inputs on the overall time-variant response, is defined and computed directly from Sobol’ sensitivity indices of the inputs at each time node. Finally, the surrogate model of the time-variant system constructed by the KMDR is directly adopted for efficient time-variant uncertainty propagation and global sensitivity analysis. Several examples demonstrate that the proposed approach can construct more accurate surrogate models and obtain more accurate time-variant uncertainty propagation and global sensitivity analysis results than existing methods with a small training set.

SpaDE: Semantic Locality Preserving Biclustering for Neuroimaging Data.

The most discriminative and revealing patterns in the neuroimaging population are often confined to smaller subdivisions of the samples and features. Especially in neuropsychiatric conditions, symptoms are expressed within micro subgroups of individuals and may only underly a subset of neurological mechanisms. As such, running a whole-population analysis yields suboptimal outcomes leading to reduced specificity and interpretability. Biclustering is a potential solution since subject heterogeneity makes one-dimensional clustering less effective in this realm. Yet, high dimensional sparse input space and semantically incoherent grouping of attributes make post hoc analysis challenging. Therefore, we propose a deep neural network called semantic locality preserving auto decoder (SpaDE), for unsupervised feature learning and biclustering. SpaDE produces coherent subgroups of subjects and neural features preserving semantic locality and enhancing neurobiological interpretability. Also, it regularizes for sparsity to improve representation learning. We employ SpaDE on human brain connectome collected from schizophrenia (SZ) and healthy control (HC) subjects. The model outperforms several state-of-the-art biclustering methods. Our method extracts modular neural communities showing significant (HC/SZ) group differences in distinct brain networks including visual, sensorimotor, and subcortical. Moreover, these bi-clustered connectivity substructures exhibit substantial relations with various cognitive measures such as attention, working memory, and visual learning.