High-dimensional Models Research Articles

Testing the number of common factors by bootstrapped sample covariance matrix in high-dimensional factor models

This paper studies the impact of bootstrap procedure on the eigenvalue distributions of the sample covariance matrix under a high-dimensional factor structure. We provide asymptotic distributions for the top eigenvalues of bootstrapped sample covariance matrix under mild conditions. After bootstrap, the spiked eigenvalues which are driven by common factors will converge weakly to Gaussian limits after proper scaling and centralization. However, the largest non-spiked eigenvalue is mainly determined by the order statistics of the bootstrap resampling weights, and follows extreme value distribution. Based on the disparate behavior of the spiked and non-spiked eigenvalues, we propose innovative methods to test the number of common factors. Indicated by extensive numerical and empirical studies, the proposed methods perform reliably and convincingly under the existence of both weak factors and cross-sectionally correlated errors. Our technical details contribute to random matrix theory on spiked covariance model with convexly decaying density and unbounded support, or with general elliptical distributions.

High-dimensional aerodynamic shape optimization framework using geometric domain decomposition and data-driven support strategy for wing design

Traditional aerodynamic shape optimization methods usually suffer from difficulties in optimization and modeling for high-dimensional aerodynamic optimization problems. To overcome these difficulties, a new high-dimensional aerodynamic shape optimization framework is proposed from the perspectives of geometric domain decomposition for dimensionality reduction and a data-driven model support strategy for changing the role of surrogate model and assisting the optimizer. Geometric domain decomposition uses proper orthogonal decomposition to transform the high-dimensional Class-Shape Function Transformation method into low-dimensional geometric modes to reduce design dimensions for parameterizing the geometric shape, which reduces optimization and modeling difficulties. Also, a data-driven support strategy is proposed to improve the effectiveness and efficiency of the optimizer by learning from historical aerodynamic data; this strategy modifies the role of surrogate model in the optimization framework to assist the optimizer; the surrogate model, serving as an auxiliary role, can reduce the dependence on model accuracy, which overcomes the difficulty of high-dimensional modeling to some extent. We applied the developed framework to transonic drag reduction of wings to validate its performance. The optimization results show that the developed framework converges faster and offers better designs compared to direct optimization and state-of-the-art surrogate-based optimization frameworks for high-dimensional aerodynamic shape optimization.

Comparing the long-term persistence of different Wolbachia strains after the release of bacteria-carrying mosquitoes

This paper proposes a bidimensional modeling framework for Wolbachia invasion, assuming imperfect maternal transmission, incomplete cytoplasmic incompatibility, and direct infection loss due to thermal stress. Our model adapts to various Wolbachia strains and retains all properties of higher-dimensional models. The conditions for the durable coexistence of Wolbachia-carrying and wild mosquitoes are expressed using the model’s parameters in a compact closed form. When the Wolbachia bacterium is locally established, the size of the remanent wild population can be assessed by a direct formula derived from the model. The model was tested for four Wolbachia strains undergoing laboratory and field trials to control mosquito-borne diseases: wMel, wMelPop, wAlbB, and wAu. As all these bacterial strains affect the individual fitness of mosquito hosts differently and exhibit different levels of resistance to temperature variations, the model helped to conclude that: (1) the wMel strain spreads faster in wild mosquito populations; (2) the wMelPop exhibits lower resilience but also guarantees the smallest size of the remanent wild population; (3) the wAlbB strain performs better at higher ambient temperatures than others; (4) the wAu strain is not sustainable and cannot persist in the wild mosquito population despite its resistance to high temperatures.

A horizontal vibration prediction method of high-speed elevator based on transferred-digital twin model

The vibrations of high-speed-elevators (HSEs) significantly impact the comfort of elevator rides, with horizontal vibrations (HV) being the most sensitive to bodies. Customized elevator products are characterized by high-speed capabilities, customization options, and complex operational conditions. These characteristics often lead to challenges such as difficulties in predicting HV, low prediction accuracy, and limited reusability of prediction models and data. In order to mitigate HV in elevators, its crucial to implement HV prediction during the design phase. To address this need, this paper proposes a method for predicting HSEHV based on a Transferred Digital Twin (TLDT) model. The HSE’s design is physically extracted, and an elevator Digital Twin (DT) model with a digital layer is constructed. This model includes product geometry models, dynamics models, and simulation models. A transfer learning (TL) model is developed to integrate simulation data generated by the digital layer with measurement data acquired from the data interface layer, resulting in high-capacity and high-fidelity DT data. Using the support vector regression (SVR) method, a high-dimensional nonlinear HSEHV prediction model is established, and is trained and optimized using DT data to achieve data-driven HSEHV prediction. The validity of this approach is confirmed using measured data from an elevator test tower. Results indicate that the SVR-based TLDT model achieves the highest accuracy and the lowest absolute error values for both peak-to-peak and A95 vibration acceleration when compared to other models such as NN, KNN, decision tree, and lasso models. Furthermore, by incorporating DT data, the model accuracies improve by 16.9%, 14.5%, 8.6%, 14.1%, and 8.6%, and 12.3%, 12.4%, 19.7%, 7.6%, and 8.7%, respectively, for the mentioned models. Finally, the HV of KLK2 elevator car system is predicted. When compared with simulation analysis and the conventional SVR prediction model, obtained results through the proposed method closely align with the measured values.

2-D probabilistic inversion of MT data and uncertainty quantification using the Hamiltonian Monte Carlo method

SUMMARY Bayesian methods provide a valuable framework for rigorously quantifying the model uncertainty arising from the inherent non-uniqueness in the magnetotelluric (MT) inversion. However, widely used Markov chain Monte Carlo (MCMC) sampling approaches usually require a significant number of model samples for accurate uncertainty estimates, making their applications computationally challenging for 2-D or 3-D MT problems. In this study, we explore the applicability of the Hamiltonian Monte Carlo (HMC) method for 2-D probabilistic MT inversion. The HMC provides a mechanism for efficient exploration in high-dimensional model space by making use of gradient information of the posterior probability distribution, resulting in a substantial reduction in the number of samples needed for reliable uncertainty quantification compared to the conventional MCMC methods. Numerical examples with synthetic data demonstrate that the HMC method achieves rapid convergence to the posterior probability distribution of model parameters with a limited number of model samples, indicating the computational advantages of the HMC in high-dimensional model space. Finally, we applied the developed approach to the COPROD2 field data set. The statistical models derived from the HMC approach agree well with previous results obtained by 2-D deterministic inversions. Most importantly, the probabilistic inversion provides valuable quantitative model uncertainty information associated with the resistivity structures derived from the observed data, which facilitates model interpretation.

Comparative analysis on feature selection process with high dimensionality through intelligent IoT sensors for hyper sensing images

Due to the high correlation between the spectral features and the noise in the spectral bands, which could have a big effect on classification performance, the classification job has usually been done at the same time as dimensionality reduction. It is known as Hughes effect or the “curse of dimensionality” in the literature when there are insufficient training examples compared to the number of spectral characteristics, which has a detrimental influence on classification performance. In this study, we compare several feature selection methods using a high-dimensional representation model through intelligent IoT sensors. Comparing it to traditional feature-selection methods in terms of classification accuracy, stability of the picked features, and computing time, it is examined using a variety of hyper spectral datasets. The findings demonstrate that the suggested method offers both high classification accuracy and reliable features with an acceptable computing time.

On lump and solitonic wave structures for the (3+1)-dimensional nonlinear evolution model

In order to understand many complex situations in wave propagation, such as heat transfer, fluid dynamics, optical fibers, electrodynamics, physics, chemistry, biology, condensed matter physics, ocean engineering, and many other branches of nonlinear science, the majority of natural processes are routinely modelled and analysed using nonlinear evolution equations. In this study, the (3+1)-dimensional nonlinear evolution equation is investigated analytically. Initially, the Hirota bilinear approach is used to develop the bilinear version of the higher dimensional nonlinear model. Consequently, we are able to design periodic wave soliton solutions, lump wave and single-kink soliton solutions, and collisions between lumps and periodic waves. Later on, the unified method is applied to develop several new travelling wave solutions for the governing model substantially. Furthermore, numerous exact solutions are analyzed graphically to explore many fascinating nonlinear dynamical structures with the aid of 3D, contour, and 2D visualizations. A variety of higher dimensional nonlinear evolution models can also be investigated by employing present approaches arising in many fields of contemporary science and technology.

Automation and value-chain embedment: evidence from Chinese manufacturing firms

ABSTRACT This study aims to explore how the digital technology revolution, brought about by industrial robots, affects the integration of Chinese firms into the global value chain. Utilizing a high-dimensional fixed-effects model, we examine matched data from the database of Chinese industrial firms and customs from 2006 to 2015. We reveal the positive impact of automation on the export domestic value-added of Chinese manufacturing firms, indicating that automation promotes the embedding of firms into the global value chain. Event study methodology finds that the unexpected increase in automation in 2009 significantly boosted firm participation in the value-chain division of labour. The role of automation in driving value-chain embedding is associated with improvements in productivity levels and reductions in relative labour production costs. This impact varies noticeably across different industries, regions, technological levels, and types of value chains, with labour-intensive industries being the most affected. Moreover, we find that automation is driving more Chinese firms to enter and continually integrate into the global value chain, although the current level of industrial automation in China has not yet reached its optimal level.

A novel kriging-improved high-dimensional model representation metamodelling technique for approximating high-dimensional problems

Aiming to address the issue of the ‘curse of dimensionality’ encountered in approximating high-dimensional problems using surrogate-based methods, high-dimensional model representation (HDMR), decomposing the high-dimensional problem into summands of different-order component functions, has been widely studied. To reduce the computational demands of the current HDMR metamodelling techniques, an improved high-dimensional model representation framework (iHDMR) is presented, taking full advantage of the relationships between the first-order and second-order component functions in the Cut-HDMR theory. then, a novel metamodelling technique, termed kriging (KRG)-iHDMR, is implemented by integrating the kriging technique and the suggested surface-axes alternating sampling strategy into the iHDMR framework. The accuracy and efficiency of KRG-iHDMR are demonstrated by various numerical and engineering examples with different dimensions. Finally, an optimization problem of the multi-stage solid launch vehicle propulsion system is introduced to verify the engineering feasibility of the KRG-iHDMR metamodelling technique.

Bernstein flows for flexible posteriors in variational Bayes

Black-box variational inference (BBVI) is a technique to approximate the posterior of Bayesian models by optimization. Similar to MCMC, the user only needs to specify the model; then, the inference procedure is done automatically. In contrast to MCMC, BBVI scales to many observations, is faster for some applications, and can take advantage of highly optimized deep learning frameworks since it can be formulated as a minimization task. In the case of complex posteriors, however, other state-of-the-art BBVI approaches often yield unsatisfactory posterior approximations. This paper presents Bernstein flow variational inference (BF-VI), a robust and easy-to-use method flexible enough to approximate complex multivariate posteriors. BF-VI combines ideas from normalizing flows and Bernstein polynomial-based transformation models. In benchmark experiments, we compare BF-VI solutions with exact posteriors, MCMC solutions, and state-of-the-art BBVI methods, including normalizing flow-based BBVI. We show for low-dimensional models that BF-VI accurately approximates the true posterior; in higher-dimensional models, BF-VI compares favorably against other BBVI methods. Further, using BF-VI, we develop a Bayesian model for the semi-structured melanoma challenge data, combining a CNN model part for image data with an interpretable model part for tabular data, and demonstrate, for the first time, the use of BBVI in semi-structured models.

Relationship of various COVID-19 antibody titer with individual characteristics and prediction of future epidemic trend in Xiamen City, China.

Reinfection of coronavirus disease 2019 (COVID-19) has raised concerns about how reliable immunity from infection and vaccination is. With mass testing for the virus halted, understanding the current prevalence of COVID-19 is crucial. This study investigated 1,191 public health workers at the Xiamen Center for Disease Control, focusing on changes in antibody titers and their relationship with individual characteristics. The study began by describing the epidemiological characteristics of the study participants. Multilinear regression (MLR) models were employed to explore the associations between individual attributes and antibody titers. Additionally, group-based trajectory models (GBTMs) were utilized to identify trajectories in antibody titer changes. To predict and simulate future epidemic trends and examine the correlation of antibody decay with epidemics, a high-dimensional transmission dynamics model was constructed. Analysis of epidemiological characteristics revealed significant differences in vaccination status between infected and non-infected groups (χ2=376.706, P<0.05). However, the distribution of antibody titers among the infected and vaccinated populations was not significantly different. The MLR model identified age as a common factor affecting titers of immunoglobulin G (IgG), immunoglobulin M (IgM), and neutralizing antibody (NAb), while other factors showed varying impacts. History of pulmonary disease and hospitalization influenced IgG titer, and factors such as gender, smoking, family history of pulmonary diseases, and hospitalization impacted NAb titers. Age was the sole determinant of IgM titers in this study. GBTM analysis indicated a "gradual decline type" trajectory for IgG (95.65%), while IgM and NAb titers remained stable over the study period. The high-dimensional transmission dynamics model predicted and simulated peak epidemic periods in Xiamen City, which correlated with IgG decay. Age-group-specific simulations revealed a higher incidence and infection rate among individuals aged 30-39 years during both the second and third peaks, followed by those aged 40-49, 50-59, 18-29, and 70-79 years. Our study shows that antibody titer could be influenced by age, previous pulmonary diseases as well as smoking. Furthermore, the decline in IgG titers is consistent with epidemic trends. These findings emphasize the need for further exploration of these factors and the development of optimized self-protection countermeasures against reinfection.

Estimation and inference for high dimensional factor model with regime switching

This paper proposes maximum (quasi)likelihood estimation for high dimensional factor models with regime switching in the loadings. The model parameters are estimated jointly by the EM (expectation maximization) algorithm, which in the current context only requires iteratively calculating regime probabilities and principal components of the weighted sample covariance matrix. When regime dynamics are taken into account, smoothed regime probabilities are calculated using a recursive algorithm. Consistency, convergence rates and limit distributions of the estimated loadings and the estimated factors are established under weak cross-sectional and temporal dependence as well as heteroscedasticity. It is worth noting that due to high dimension, regime switching can be identified consistently after the switching point with only one observation period. Simulation results show good performance of the proposed method. An application to the FRED-MD dataset illustrates the potential of the proposed method for detection of business cycle turning points.

Hybrid unadjusted Langevin methods for high-dimensional latent variable models

The exact estimation of latent variable models with big data is known to be challenging. The latents have to be integrated out numerically, and the dimension of the latent variables increases with the sample size. This paper develops a novel approximate Bayesian method based on the Langevin diffusion process. The method employs the Fisher identity to integrate out the latent variables, which makes it accurate and computationally feasible when applied to big data. In contrast to other approximate estimation methods, it does not require the choice of a parametric distribution for the unknowns, which often leads to inaccuracies. In an empirical discrete choice example with a million observations, the proposed method accurately estimates the posterior choice probabilities using only 2% of the computation time of exact MCMC.

Early stopping for L2-boosting in high-dimensional linear models

Early stopping for L2-boosting in high-dimensional linear models

Group Variable Selection via Group Sparse Neural Network

Group variable selection is an important issue in high-dimensional data modeling and most of existing methods consider only the linear model. Therefore, a new method based on the deep neural network (DNN), an increasingly popular nonlinear method in both statistics and deep learning communities, is proposed. The method is applicable to general nonlinear models, including the linear model as a special case. Specifically, a group sparse neural network (GSNN) is designed, where the definition of nonlinear group high-level features (NGHFs) is generalized to the network structure. A two-stage group sparse (TGS) algorithm is employed to induce group variables selection by performing group structure selection on the network. GSNN is promising for complex nonlinear systems with interactions and correlated predictors, overcoming the shortcomings of linear or marginal variable selection methods. Theoretical results on convergence and group-level selection consistency are also given. Simulations results and real data analysis demonstrate the superiority of our method.

An integrated molecular risk score early in life for subsequent childhood asthma risk.

Numerous children present with early wheeze symptoms, yet solely a subgroup develops childhood asthma. Early identification of children at risk is key for clinical monitoring, timely patient-tailored treatment, and preventing chronic, severe sequelae. For early prediction of childhood asthma, we aimed to define an integrated risk score combining established risk factors with genome-wide molecular markers at birth, complemented by subsequent clinical symptoms/diagnoses (wheezing, atopic dermatitis, food allergy). Three longitudinal birth cohorts (PAULINA/PAULCHEN, n = 190 + 93 = 283, PASTURE, n = 1133) were used to predict childhood asthma (age 5-11) including epidemiological characteristics and molecular markers: genotype, DNA methylation and mRNA expression (RNASeq/NanoString). Apparent (ap) and optimism-corrected (oc) performance (AUC/R2) was assessed leveraging evidence from independent studies (Naïve-Bayes approach) combined with high-dimensional logistic regression models (LASSO). Asthma prediction with epidemiological characteristics at birth (maternal asthma, sex, farm environment) yielded an ocAUC = 0.65. Inclusion of molecular markers as predictors resulted in an improvement in apparent prediction performance, however, for optimism-corrected performance only a moderate increase was observed (upto ocAUC = 0.68). The greatest discriminate power was reached by adding the first symptoms/diagnosis (up to ocAUC = 0.76; increase of 0.08, p = .002). Longitudinal analysis of selected mRNA expression in PASTURE (cord blood, 1, 4.5, 6 years) showed that expression at age six had the strongest association with asthma and correlation of genes getting larger over time (r = .59, p < .001, 4.5-6 years). Applying epidemiological predictors alone showed moderate predictive abilities. Molecular markers from birth modestly improved prediction. Allergic symptoms/diagnoses enhanced the power of prediction, which is important for clinical practice and for the design of future studies with molecular markers.

Artificial neural networks to solve dynamic programming problems: A bias-corrected Monte Carlo operator

Artificial Neural Networks (ANNs) are powerful tools that can solve dynamic programming problems arising in economics. In this context, estimating ANN parameters involves minimizing a loss function based on the model's stochastic functional equations. In general, the expectations appearing in the loss function admit no closed-form solution, so numerical approximation techniques must be used. In this paper, I analyze a bias-corrected Monte Carlo operator (bc-MC) that approximates expectations by Monte Carlo. I show that the bc-MC operator is a generalization of the all-in-one expectation operator, already proposed in the literature. I demonstrate that, under some conditions on the primitives of the economic model, the bc-MC operator is the unbiased estimator of the loss function with the minimum variance. I propose a method to optimally set the hyperparameters defining the bc-MC operator, and illustrate the findings numerically with well-known economic models. I also demonstrate that the bc-MC operator can scale to high-dimensional models. With just approximately a minute of computing time, I find a global solution to an economic model with a kink in the decision function and more than 100 dimensions.

Variational Bayesian Variable Selection for High-Dimensional Hidden Markov Models

The Hidden Markov Model (HMM) is a crucial probabilistic modeling technique for sequence data processing and statistical learning that has been extensively utilized in various engineering applications. Traditionally, the EM algorithm is employed to fit HMMs, but currently, academics and professionals exhibit augmenting enthusiasm in Bayesian inference. In the Bayesian context, Markov Chain Monte Carlo (MCMC) methods are commonly used for inferring HMMs, but they can be computationally demanding for high-dimensional covariate data. As a rapid substitute, variational approximation has become a noteworthy and effective approximate inference approach, particularly in recent years, for representation learning in deep generative models. However, there has been limited exploration of variational inference for HMMs with high-dimensional covariates. In this article, we develop a mean-field Variational Bayesian method with the double-exponential shrinkage prior to fit high-dimensional HMMs whose hidden states are of discrete types. The proposed method offers the advantage of fitting the model and investigating specific factors that impact the response variable changes simultaneously. In addition, since the proposed method is based on the Variational Bayesian framework, the proposed method can avoid huge memory and intensive computational cost typical of traditional Bayesian methods. In the simulation studies, we demonstrate that the proposed method can quickly and accurately estimate the posterior distributions of the parameters with good performance. We analyzed the Beijing Multi-Site Air-Quality data and predicted the PM2.5 values via the fitted HMMs.

Efficient variable selection for high-dimensional multiplicative models: a novel LPRE-based approach

Efficient variable selection for high-dimensional multiplicative models: a novel LPRE-based approach

Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model

Text-guided motion synthesis aims to generate 3D human motion that not only precisely reflects the textual description but reveals the motion details as much as possible. Pioneering methods explore the diffusion model for text-to-motion synthesis and obtain significant superiority. However, these methods conduct diffusion processes either on the raw data distribution or the low-dimensional latent space, which typically suffer from the problem of modality inconsistency or detail-scarce. To tackle this problem, we propose a novel Basic-to-Advanced Hierarchical Diffusion Model, named B2A-HDM, to collaboratively exploit low-dimensional and high-dimensional diffusion models for high quality detailed motion synthesis. Specifically, the basic diffusion model in low-dimensional latent space provides the intermediate denoising result that to be consistent with the textual description, while the advanced diffusion model in high-dimensional latent space focuses on the following detail-enhancing denoising process. Besides, we introduce a multi-denoiser framework for the advanced diffusion model to ease the learning of high-dimensional model and fully explore the generative potential of the diffusion model. Quantitative and qualitative experiment results on two text-to-motion benchmarks (HumanML3D and KIT-ML) demonstrate that B2A-HDM can outperform existing state-of-the-art methods in terms of fidelity, modality consistency, and diversity.