Modeling High-dimensional Data Research Articles

Functional Neural Networks for High-Dimensional Genetic Data Analysis.

Artificial intelligence (AI) is a thriving research field with many successful applications in areas such as computer vision and speech recognition. Machine learning methods, such as artificial neural networks (ANN), play a central role in modern AI technology. While ANN also holds great promise for human genetic research, the high-dimensional genetic data and complex genetic structure bring tremendous challenges. The vast majority of genetic variants on the genome have small or no effects on diseases, and fitting ANN on a large number of variants without considering the underlying genetic structure (e.g., linkage disequilibrium) could bring a serious overfitting issue. Furthermore, while a single disease phenotype is often studied in a classic genetic study, in emerging research fields (e.g., imaging genetics), researchers need to deal with different types of disease phenotypes. To address these challenges, we propose a functional neural networks (FNN) method. FNN uses a series of basis functions to model high-dimensional genetic data and a variety of phenotype data and further builds a multi-layer functional neural network to capture the complex relationships between genetic variants and disease phenotypes. Through simulations, we demonstrate the advantages of FNN for high-dimensional genetic data analysis in terms of robustness and accuracy. The real data applications also showed that FNN attained higher accuracy than the existing methods.

Group Variable Selection via Group Sparse Neural Network

Group variable selection is an important issue in high-dimensional data modeling and most of existing methods consider only the linear model. Therefore, a new method based on the deep neural network (DNN), an increasingly popular nonlinear method in both statistics and deep learning communities, is proposed. The method is applicable to general nonlinear models, including the linear model as a special case. Specifically, a group sparse neural network (GSNN) is designed, where the definition of nonlinear group high-level features (NGHFs) is generalized to the network structure. A two-stage group sparse (TGS) algorithm is employed to induce group variables selection by performing group structure selection on the network. GSNN is promising for complex nonlinear systems with interactions and correlated predictors, overcoming the shortcomings of linear or marginal variable selection methods. Theoretical results on convergence and group-level selection consistency are also given. Simulations results and real data analysis demonstrate the superiority of our method.

Economic uncertainty and stock market asymmetric volatility: analysis based on the asymmetric GARCH-MIDAS model

PurposeWe aim to clarify the effect of economic uncertainty on Chinese stock market fluctuations. We extend the understanding of the asymmetric connectedness between economic uncertainty and stock market risk and provide different characteristics of spillovers from economic uncertainty to both upside and downside risk. Furthermore, we aim to provide the different impact patterns of stock market volatility following several exogenous shocks.Design/methodology/approachWe construct a Chinese economic uncertainty index using a Factor-Augmented Variable Auto-Regressive Stochastic Volatility (FAVAR-SV) model for high-dimensional data. We then examine the asymmetric impact of realized volatility and economic uncertainty on the long-term volatility components of the stock market through the asymmetric Generalized Autoregressive Conditional Heteroskedasticity-Mixed Data Sampling (GARCH-MIDAS) model.FindingsNegative news, including negative return-related volatility and higher economic uncertainty, has a greater impact on the long-term volatility components than positive news. During the financial crisis of 2008, economic uncertainty and realized volatility had a significant impact on long-term volatility components but did not constitute long-term volatility components during the 2015 A-share stock market crash and the 2020 COVID-19 pandemic. The two-factor asymmetric GARCH-MIDAS model outperformed the other two models in terms of explanatory power, fitting ability and out-of-sample forecasting ability for the long-term volatility component.Research limitations/implicationsMany GARCH series models can also combine the GARCH series model with the MIDAS method, including but not limited to Exponential GARCH (EGARCH) and Threshold GARCH (TGARCH). These diverse models may exhibit distinct reactions to economic uncertainty. Consequently, further research should be undertaken to juxtapose alternative models for assessing the stock market response.Practical implicationsOur conclusions have important implications for stakeholders, including policymakers, market regulators and investors, to promote market stability. Understanding the asymmetric shock arising from economic uncertainty on volatility enables market participants to assess the potential repercussions of negative news, engage in timely and effective volatility prediction, implement risk management strategies and offer a reference for financial regulators to preemptively address and mitigate systemic financial risks.Social implicationsFirst, in the face of domestic and international uncertainties and challenges, policymakers must increase communication with the market and improve policy transparency to effectively guide market expectations. Second, stock market authorities should improve the basic regulatory system of the capital market and optimize investor structure. Third, investors should gradually shift to long-term value investment concepts and jointly promote market stability.Originality/valueThis study offers a novel perspective on incorporating a Chinese economic uncertainty index constructed by a high-dimensional FAVAR-SV model into the asymmetric GARCH-MIDAS model.

Mining the colossal patterns using ISSA based KMC with VGHHO clustering model for high dimensional data

High-dimensional datasets comprise a few rows but a huge total of features. The increased difficulty data scientists have in efficiently gleaning insights from high-dimensional large data is a direct outcome of this trend. Therefore, a K-Means clustering method (KMC) must be implemented to cleanse this data, with the centroid of KMC being ideally chosen by improved squirrel search algorithm (ISSA). The suggested algorithm features not one but two different types of searches: the leaping search and the progressive search. The linear regression selection strategy is utilised to automatically choose the applicable approach during the evolutionary phase; this increases SSA's stability. Harris Hawks Optimizer (HHO), which replicates the behaviour of a Harris hawk during rabbit predation, is applied to the cleaned data to cluster the massive pattern. However, HHO has problems with low accuracy and early convergence because of its inability to strike a good balance among exploitation. A new variant of HHO, dubbed velocity-guided HHO (VGHHO), including three improvements is proposed to address these drawbacks. By including a velocity operator and an inertia weight into the search equation, we are able to create a unique modified position search equation for use during the exploitation phase. Then, we incorporate a learning mechanism based on refraction and opposition to provide the promising resolutions and aid the swarm in escaping the local optimal solution. After the massive cluster has been built with VGHHO, uproot technology is used to uncover massive patterns. We run the tests on a wide variety of high-dimensional datasets and employ a number of different efficiency metrics. Evidence from these investigations shows that the proposed method produces high-quality mining results.

A Bayesian Gaussian Process-Based Latent Discriminative Generative Decoder (LDGD) Model for High-Dimensional Data

A Bayesian Gaussian Process-Based Latent Discriminative Generative Decoder (LDGD) Model for High-Dimensional Data

123 Benchmarking of Artificial Neural Network Models for Genomic Prediction of Quantitative Traits in Pigs

Abstract Artificial neural networks (ANN) are a type of machine learning model that has been applied to various genomic problems, with the ability to learn non-linear relationships and model high-dimensional data. ANNs also have the potential in genomic prediction by capturing the intricate relationship between genetic variants and phenotypes. However, there is currently a limited effort to investigate the performance and feasibility of ANNs for pig genomic predictions. In this study, we evaluated the predictive performance of TensorFlow’s ANN models with one-layer, two-layer, and three-layer structures (with zero, one, and two hidden layers, respectively), in comparison with five linear methods, including GBLUP, LDAK, BayesR, SLEMM and scikit-learn’s ridge regression using data of six quantitative traits including off-test body weight (WT), off-test back fat thickness (BF), off-test loin muscle depth (MS), number of piglets born alive (NBA), number of piglets born dead (NBD), and number of piglets weaned (NW). Furthermore, we assessed the computational efficiency of ANNs on both CPU and GPU. The benchmarking was based on cross-validations of 26,190 genotyped pigs. We employed hyperband tuning to optimize the hyper-parameters and select the best model among one-layer, two-layer, and three-layer structures. Results showed that the one-layer structure, which is equivalent to ridge regression, yielded the best performance comparable to that of GBLUP. Using the optimal hyper-parameters for two-layer and three-layer structures, ANNs underperformed GBLUP in terms of accuracy. Of the five linear methods, BayesR and SLEMM performed similarly and the best, followed by LDAK, scikit-learn’s ridge regression, and GBLUP. Moreover, SLEMM was the fastest, which completed training with 21k individuals and 30k SNPs in 2.6 minutes. Compared with CPUs, GPUs exhibited a comparable computational speed for one-layer ANN but offered significant gains in computational efficiency for multi-layer ANNs. Based on our analysis of optimal hyper-parameters for two-layer ANN with BF, we found that using a GPU can lead to a five-fold increase in processing speed compared with using a conventional CPU, but it is still slower than GBLUP. In addition, hyper-parameter tuning (particularly for L2 regularization and the number of dense units in hidden layers) is critical for improving the genomic prediction accuracy in pigs. In conclusion, we found ANN with up to three layers could not improve genomic predictions compared with routine linear methods for pig quantitative traits.

XAI SHAP 기반 토지 피복 구성에 따른 지표면 온도의 영향 분석

Recently, with the development of artificial intelligence architecture and the rapid improvement of hardware performance, various artificial intelligence models are attracting attention and their usefulness are increasing. However, unlike the rate of development of performance, the development of model's explanatory ability is slow to progress. The increasingly complex decision branching points of AI models are increasing exponentially, which hinders researchers' ease of model interpretation. Explainable Artificial Intelligence, XAI has emerged to solve the above complexity, and helps to improve interpretability and reliability by decomposing the black box of the model to a level that researchers can understand. This study estimates the surface temperature of a specific spatial unit using land cover data of a cell unit in Korea, Seoul. The limitations of the existing statistical model for high-dimensional data were examined, and the estimation results were compared using a machine learning model. Models used include Lasso regression(Least absolute linkage and selection operator), random forest regression, and XGBoost (eXtreme Gradient Boosting) regression. Finally, based on the fitted result of the XGBoost regression model, XAI SHAP was carried out.

Two-way dynamic factor models for high-dimensional matrix-valued time series

Abstract In this article, we introduce a two-way dynamic factor model (2w-DFM) for high-dimensional matrix-valued time series and study some of the basic theoretical properties in terms of identifiability and estimation accuracy. The proposed model aims to capture separable and low-dimensional effects of row and column attributes and their correlations across rows, columns, and time points. Complementary to other dynamic factor models for high-dimensional data, the 2w-DFM inherits the dimension-reduction feature of factor models but assumes additive row and column factors for easier interpretability. We provide conditions to ensure model identifiability and consider a quasi-likelihood based two-step method for parameter estimation. Under an asymptotic regime where the size of the data matrices as well as the length of the time series increase, we establish that the estimators achieve the optimal rate of convergence and are asymptotically normal. The asymptotic properties are reaffirmed empirically through simulation studies. An application to air quality data in Chinese cities is given to illustrate the merit of the 2w-DFM.

Research and Application Analysis of Correlative Optimization Algorithms for GAN

Generative Adversarial Networks (GANs) have been one of the most successful deep learning architectures in recent years, providing a powerful way to model high-dimensional data such as images, audio, and text data. GANs use two neural networks, generator and discriminator, to generate samples that resemble real data. The generator tries to create realistic looking samples while the discriminator tries to differentiate the generated samples from real ones. Through this adversarial training process, the generator learns to produce high-quality samples indistinguishable from the real ones.Different optimization algorithms have been utilized in GAN research, including different types of loss functions and regularization techniques, to improve the performance of GANs. Some of the most significant recent developments in GANs include M-DCGAN, which stands for multi-scale deep convolutional generative adversarial network, designed for image dataset augmentation; StackGAN, which is a text-to-image generation technique designed to produce high-resolution images with fine details and BigGAN, a scaled-up version of GAN that has shown improved performance in generating high-fidelity images.Moreover, the potential applications of GANs are vast and cross-disciplinary. They have been applied in various fields such as image and video synthesis, data augmentation, image translation, and style transfer. GANs also show promise in extending their use to healthcare, finance, and creative art fields. Despite their significant advancements and promising applications, GANs face several challenges such as mode collapse, vanishing gradients, and instability, which need to be addressed to achieve better performance and broader applicability.In conclusion, this review gives insights into the current state-of-the-art in GAN research, discussing its core ideas, structure, optimization techniques, applications, and challenges faced. This knowledge aims to help researchers and practitioners alike to understand the current GAN models' strengths and weaknesses and guide future GAN developments. As GANs continue to evolve, they have the potential to transform the way we understand and generate complex datasets across various fields.

Semi-supervised classifier ensemble model for high-dimensional data

To complete the challenging task of high-dimensional data classification with limited labeled samples, we propose two semi-supervised learning models, namely the random subspace classifier ensemble model (SSRS) and its adaptive version (ASSRS). Considering the unique characteristics of high-dimensional data, SSRS selects subspaces of sample and feature dimensions and then reduces the dimensions of each subspace. To improve SSRS performance further, we designed a sample-labeling auxiliary algorithm, adaptive sample subspace algorithm, and adaptive weight voting rule for ASSRS to increase the proportion of labeled samples, obtain a suitable sample subspace for each feature subspace, and acquire a relative optimal weight for each base classifier. Experiments revealed that the performances of SSRS and ASSRS were better than those of other competitive algorithms and that the performance of ASSRS was stronger than that of SSRS. Additionally, we can accurately label samples in datasets where the proportion of labeled samples is relatively low by using SSRS and ASSRS. Because analysts are facing large numbers of high-dimensional datasets with limited labels, it is important to make accurate predictions based on a limited proportion of labeled data.

A Time Series Data Regression Model Based on Multi-kernel KPCA Dimension Reduction and Bagging Algorithms

Regression models for high-dimensional data have always been a hot topic in the field of statistical learning. Considering the case that the predictor variable is a high-frequency time series and the response variable is a continuous scalar, this paper proposes a regression method based on a multi-kernel KPCA Dimension reduction method and the Bagging algorithms. The proposed method adaptively solves the problem of kernel function selection and unsupervised ness in KPCA Dimension reduction by splicing the projection data under various kernel functions and using the Bagging algorithms to mine the relationship between projection data and the response variable. In the real data analysis, this paper selects the multiple regression model, the LASSO regression based on model selection, the multiple regression model based on PCA Dimension reduction and single-kernel KPCA Dimension reduction as comparison methods. The results show that the proposed method has better performance than other comparison methods. Since the basic regressor in the Bagging framework is model-free, some prediction models can be used to improve the prediction accuracy for more complex data situations.

Modeling in higher dimensions to improve diagnostic testing accuracy: Theory and examples for multiplex saliva-based SARS-CoV-2 antibody assays.

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has emphasized the importance and challenges of correctly interpreting antibody test results. Identification of positive and negative samples requires a classification strategy with low error rates, which is hard to achieve when the corresponding measurement values overlap. Additional uncertainty arises when classification schemes fail to account for complicated structure in data. We address these problems through a mathematical framework that combines high dimensional data modeling and optimal decision theory. Specifically, we show that appropriately increasing the dimension of data better separates positive and negative populations and reveals nuanced structure that can be described in terms of mathematical models. We combine these models with optimal decision theory to yield a classification scheme that better separates positive and negative samples relative to traditional methods such as confidence intervals (CIs) and receiver operating characteristics. We validate the usefulness of this approach in the context of a multiplex salivary SARS-CoV-2 immunoglobulin G assay dataset. This example illustrates how our analysis: (i) improves the assay accuracy, (e.g. lowers classification errors by up to 42% compared to CI methods); (ii) reduces the number of indeterminate samples when an inconclusive class is permissible, (e.g. by 40% compared to the original analysis of the example multiplex dataset) and (iii) decreases the number of antigens needed to classify samples. Our work showcases the power of mathematical modeling in diagnostic classification and highlights a method that can be adopted broadly in public health and clinical settings.

Robust tensor-on-tensor regression for multidimensional data modeling

In recent years, high-dimensional data, such as waveform signals and images have become ubiquitous. This type of data is often represented by multiway arrays or tensors. Several statistical models, including tensor regression, have been developed for such tensor data. However, these models are sensitive to the presence of arbitrary outliers within the tensors. To address the issue, this article proposes a Robust Tensor-On-Tensor (RTOT) regression approach, which has the capability of modeling high-dimensional data when the data is corrupted by outliers. Through several simulations and case studies, we evaluate the performance of the proposed method. The results reveal the advantage of the RTOT over some benchmarks in the literature in terms of estimation error. A Python implementation is available at https://github.com/Reisi-Lab/RTOT.git.

One-step sparse estimates in the reverse penalty for high-dimensional correlated data

Variable selection methods are widely used in modeling high-dimensional data, such as portfolios, gene selection, etc. But strong correlations exist in high-dimensional data leading to poor model estimation and prediction. In this paper, inspired by Yang and Yang (2021), we propose generalized adaptive smooth adjustment for correlated effects estimator to deal with high-dimensional correlated data, reducing estimation bias and encouraging grouping, but also improving the correct rate of model identification. Furthermore, we use the one-step estimation algorithm to convert the non-convex penalty function to L1 penalty, greatly reducing the computational cost. Finally, we give some finite sample performance and case analysis of the proposed method and demonstrate the consistency of variable selection and oracle property under some conditions.

Digital Health Transformers and Opportunities for Artificial Intelligence-Enabled Nephrology.

Digital Health Transformers and Opportunities for Artificial Intelligence-Enabled Nephrology.

Big problems in spatio-temporal disease mapping: Methods and software

Background and objective: Fitting spatio-temporal models for areal data is crucial in many fields such as cancer epidemiology. However, when data sets are very large, many issues arise. The main objective of this paper is to propose a general procedure to analyze high-dimensional spatio-temporal areal data, with special emphasis on mortality/incidence relative risk estimation.Methods: We present a pragmatic and simple idea that permits hierarchical spatio-temporal models to be fitted when the number of small areas is very large. Model fitting is carried out using integrated nested Laplace approximations over a partition of the spatial domain. We also use parallel and distributed strategies to speed up computations in a setting where Bayesian model fitting is generally prohibitively time-consuming or even unfeasible.Results:Using simulated and real data, we show that our method outperforms classical global models. We implement the methods and algorithms that we develop in the open-source R package bigDM where specific vignettes have been included to facilitate the use of the methodology for non-expert users.Conclusions: Our scalable methodology proposal provides reliable risk estimates when fitting Bayesian hierarchical spatio-temporal models for high-dimensional data.

Interaction Modelling of High Dimensional Production Data

The final goal of this study is to present a practical interaction modeling methodology for big and complex industrial data in which there are mixed difficulty of high dimension, sparsity, interaction, multicollinearity, series bias, low signal noise ratio, nonlinearity, and so on. Particularly, this paper focuses on the issues of high dimension, sparsity, and interactions. In addition, it is necessary to estimate unknown factors and interactions in order to improve overall efficiency in complex and diverse situations of industry. The main purpose here is to summarize the applicability and challenges of Factorization Machines (FMs) to manufacturing data. FMs has possibility because of the ad-vantage of to identify unknown factors and its robustness to data with more than 95% missing values which has already been reported for recommendation problems. Manufacturing data assumed here incudes the inspected quality of the final products, and a set of various variables (settings, sensing results, etc.) of the series production process which may relate to the quality. Common characteristics of manufacturing data include mixture of numerical and categorical variables, high dimensionality, many missing values, and so on. Therefore, this paper particularly focuses on sparse high-dimensional type-mix data. So far, while interaction modeling of high-dimensional data has been discussed and verified, mainly for screening methods, still there is very few studies to apply FMs to the manufacturing data with many numerical variables. There-fore, we report the evaluation results using some synthesis data and simulation data from actual oracle model to summarize FMs applicability as the first step. The second step is to compare the performance of sparse FMs which has feature selection function, beside our proposal.

High-dimensional aerodynamic data modeling using a machine learning method based on a convolutional neural network

Modeling high-dimensional aerodynamic data presents a significant challenge in aero-loads prediction, aerodynamic shape optimization, flight control, and simulation. This article develops a machine learning approach based on a convolutional neural network (CNN) to address this problem. A CNN can implicitly distill features underlying the data. The number of parameters to be trained can be significantly reduced because of its local connectivity and parameter-sharing properties, which is favorable for solving high-dimensional problems in which the training cost can be prohibitive. A hypersonic wing similar to the Sanger aerospace plane carrier wing is employed as the test case to demonstrate the CNN-based modeling method. First, the wing is parameterized by the free-form deformation method, and 109 variables incorporating flight status and aerodynamic shape variables are defined as model input. Second, more than 7000 sample points generated by the Latin hypercube sampling method are evaluated by performing computational fluid dynamics simulations using a Reynolds-averaged Navier–Stokes flow solver to obtain an aerodynamic database, and a CNN model is built based on the observed data. Finally, the well-trained CNN model considering both flight status and shape variables is applied to aerodynamic shape optimization to demonstrate its capability to achieve fast optimization at multiple flight statuses.

Minimum standards for evaluating machine-learned models of high-dimensional data.

The maturation of machine learning and technologies that generate high dimensional data have led to the growth in the number of predictive models, such as the “epigenetic clock”. While powerful, machine learning algorithms run a high risk of overfitting, particularly when training data is limited, as is often the case with high-dimensional data (“large p, small n”). Making independent validation a requirement of “algorithmic biomarker” development would bring greater clarity to the field by more efficiently identifying prediction or classification models to prioritize for further validation and characterization. Reproducibility has been a mainstay in science, but only recently received attention in defining its various aspects and how to apply these principles to machine learning models. The goal of this paper is merely to serve as a call-to-arms for greater rigor and attention paid to newly developed models for prediction or classification.

It's all relative: Regression analysis with compositional predictors.

Compositional data reside in a simplex and measure fractions or proportions of parts to a whole. Most existing regression methods for such data rely on log-ratio transformations that are inadequate or inappropriate in modeling high-dimensional data with excessive zeros and hierarchical structures. Moreover, such models usually lack a straightforward interpretation due to the interrelation between parts of a composition. We develop a novel relative-shift regression framework that directly uses proportions as predictors. The new framework provides a paradigm shift for regression analysis with compositional predictors and offers a superior interpretation of how shifting concentration between parts affects the response. New equi-sparsity and tree-guided regularization methods and an efficient smoothing proximal gradient algorithm are developed to facilitate feature aggregation and dimension reduction in regression. A unified finite-sample prediction error bound is derived for the proposed regularized estimators. We demonstrate the efficacy of the proposed methods in extensive simulation studies and a real gut microbiome study. Guided by the taxonomy of the microbiome data, the framework identifies important taxa at different taxonomic levels associated with the neurodevelopment of preterminfants.