High-dimensional Models Research Articles

Multichannel Wavelet Kernel Network for High Dimensional Inverse Modeling of Microwave Filters

This paper proposes a multichannel wavelet kernel network (MWKN) modeling technique with a two-stage training technique for high-dimensional inverse modeling of microwave filters. The real and imaginary parts of the transmission and reflection characteristics are used as the model inputs, while the geometric parameters of the filter are designated as the outputs. Since the electrical signal in microwave inverse modeling encompasses multiple frequency components and complex information arising from the subtle dimensional changes in the metal pattern, the wavelet transform is introduced by leveraging its powerful multi-scale and approximate detail features to form the discrete wavelet convolution layer in the proposed MWKN. To adapt to the learning of approximate detailed features at different scales, the learnable parameters of this layer and the weights of the backbone network are adjusted in stages through a two-stage training strategy based on particle swarm optimization (PSO), which jointly promotes the rapid convergence of the model. Three numerical examples demonstrate the effectiveness and robustness of the proposed MWKN model. Compared with the traditional design method using electromagnetic (EM) simulation, this approach significantly and substantially reduces the repeated calculation time and is capable of predicting the geometry that meets the design specifications within 0.42 s.

A robust and efficient change point detection method for high-dimensional linear models

In the context of linear models, a key problem of interest is to estimate the regression coefficient. Nevertheless, in certain instances, the vector of unknown coefficient parameters in a linear regression model differs from one segment to another. In this paper, when the dimension of covariates is high, a new method is proposed to examine a linear model in which the regression coefficient of two subpopulations may be different. To achieve robustness and efficiency, we introduce modal linear regression as a means of estimating the unknown coefficient parameters. Furthermore, our proposed method is capable of selecting variables and checking change points. Under certain mild assumptions, the limiting behavior of our proposed method can be established. Additionally, an estimation algorithm based on kick-one-off and SCAD approach is developed to implement in practice. For illustration, simulation studies and a real data are considered to assess the performance of our proposed method.

Stabilizing training of affine coupling layers for high-dimensional variational inference

Abstract Variational inference with normalizing flows is an increasingly popular alternative to MCMC methods. In particular, normalizing flows based on affine coupling layers (Real NVPs) are frequently used due to their good empirical performance. In theory, increasing the depth of normalizing flows should lead to more accurate posterior approximations. However, in practice, training deep normalizing flows for approximating high-dimensional posterior distributions is often infeasible due to the high variance of the stochastic gradients. 
In this work, we show that previous methods for stabilizing the variance of stochastic gradient descent can be insufficient to achieve stable training of Real NVPs. As the source of the problem, we identify that, during training, samples often exhibit unusual high values. 
As a remedy, we propose a combination of two methods: (1) soft-thresholding of the scale in Real NVPs, and (2) a bijective soft log transformation of the samples. We evaluate these and other previously proposed modification on several challenging target distributions, including a high-dimensional horseshoe logistic regression model. Our experiments show that with our modifications, stable training of Real NVPs for posteriors with several thousand dimensions and heavy tails is possible, allowing for more accurate marginal likelihood estimation via importance sampling. 
Moreover, we evaluate several common training techniques and architecture choices and provide practical advise for training Real NVPs for high-dimensional variational inference. Finally, we also provide new empirical and theoretical justification that optimizing the evidence lower bound of normalizing flows leads to good posterior distribution coverage.

Polyglot entrainment for higher dimensional neuronal models

The entrainment of biological oscillators is a classic problem in the field of dynamical systems and synchronization. This paper explores a novel type of entrainment mechanism referred to as polyglot entrainment [Khan et al., “The emergence of polyglot entrainment responses to periodic inputs in vicinities of Hopf bifurcations in slow–fast systems,” Chaos 32, 063137 (2022)] (multiple disconnected 1:1 regions for a range of forcing amplitude) for higher dimensional nonlinear systems. Polyglot entrainment has been recently explored only in two-dimensional slow–fast models in the vicinity of Hopf bifurcations (HBs). Heading toward generality, in this research, we investigate the phenomenon of polyglot entrainment in higher-dimensional conductance-based models including the four-dimensional Hodgkin-Huxley model and its reduced three-dimensional version. We utilize dynamical systems tools to uncover the mechanism of entrainment and geometric structure of the null surfaces to explore the conditions for the existence of polyglot entrainment in these models. In light of our findings, in the vicinity of HB, when an unforced system acts as a damped oscillator and the fixed point is located near a cubic-like manifold, polyglot entrainment is observed.

Partially factorized variational inference for high-dimensional mixed models

Abstract While generalized linear mixed models are a fundamental tool in applied statistics, many specifications, such as those involving categorical factors with many levels or interaction terms, can be computationally challenging to estimate due to the need to compute or approximate high-dimensional integrals. Variational inference is a popular way to perform such computations, especially in the Bayesian context. However, naive use of such methods can provide unreliable uncertainty quantification. We show that this is indeed the case for mixed models, proving that standard mean-field variational inference dramatically underestimates posterior uncertainty in high-dimensions. We then show how appropriately relaxing the mean-field assumption leads to methods whose uncertainty quantification does not deteriorate in high-dimensions, and whose total computational cost scales linearly with the number of parameters and observations. Our theoretical and numerical results focus on mixed models with Gaussian or binomial likelihoods, and rely on connections to random graph theory to obtain sharp high-dimensional asymptotic analysis. We also provide generic results, which are of independent interest, relating the accuracy of variational inference to the convergence rate of the corresponding coordinate ascent algorithm that is used to find it. Our proposed methodology is implemented in the R package vglmer. Numerical results with simulated and real data examples illustrate the favourable computation cost versus accuracy trade-off of our approach compared to various alternatives.

Two dimensional computational model of calcium and IP3 interacting system dynamics in an obese hepatocyte cell

Abstract Calcium ion (Ca2+) signaling is crucial in regulating numerous cellular processes vital for preserving structural integrity and functional equilibrium across diverse cell types. Both the calcium stores and mitochondria play significant roles in this signaling pathway. The calcium source may be in the form of a blip or a puff depending on the various conditions of the cellular systems. The one dimensional model of calcium dynamics with IP3 gives crucial insight of feedback mechanisms influencing calcium homeostasis. In order to obtain deeper insights of local impacts of various mechanisms and feedbacks in hepatocyte cell, it is necessary to develop the models in higher dimensions. In order to get more deeper insights, two dimensional model is proposed assuming the phenomena to be uniform along z dimension. This research presents a two-dimensional computational model to analyse the interactive system dynamics of inositol 1,4,5-trisphosphate (IP3) and Ca2+, aiming to assess how these signaling patterns influence hepatocyte functionality which allows to incorporate puff type of calcium source under both obese and normal physiological states. It further examines the implications of calcium signaling on NADH synthesis, ATP production, and degradation rates. Numerical simulations are executed utilising the Crank-Nicolson method for temporal analysis and the Linear Finite Element Method for spatial analysis. Additionally, the study conducts a comparative analysis of calcium signaling between obese and normal hepatocyte. The findings offer enhanced insights into the interactive system dynamics of IP3 and Ca2+ in hepatocytes, elucidating the effects of various parameter alterations on cellular behaviour in both states.

Soil Moisture-Cloud-Precipitation Feedback in the Lower Atmosphere From Functional Decomposition of Satellite Observations.

The feedback of topsoil moisture (SM) content on convective clouds and precipitation is not well understood and represented in the current generation of weather and climate models. Here, we use functional decomposition of satellite-derived SM and cloud vertical profiles (CVP) to quantify the relationship between SM and the vertical distribution of cloud water in the central US. High-dimensional model representation is used to disentangle the contributions of SM and other land-surface and atmospheric variables to the CVP. Results show that the sign and strength of the SM-cloud-precipitation feedback varies with cloud height and time lag and displays a large spatial variability. Positive anomalies in antecedent 7-hr SM and land-surface temperature enhance cloud reflectivity up to 4 dBZ in the lower atmosphere about 1-3km above the surface. Our approach presents new insights into the SM-cloud-precipitation feedback and aids in the diagnosis of land-atmosphere interactions simulated by weather and climate models.

Modeling the extra column volume of a micro simulated moving bed chromatography system: Introducing the equivalent radial flow rate distribution

Recently, the focus in chromatography model development has expanded to include the modeling of extra column volume (ECV), particularly in small- and lab-scale systems where ECV can constitute a significant portion of the total volume. Typically, ECV is modeled with 1D approaches, for example with combinations of dispersed plug flow reactors (DPFRs) and continuously stirred tank reactors (CSTRs). However, radial inhomogeneities in the ECV concentration profile necessitate higher-dimensional models for more accurate predictions. Searching for a suitable modeling approach for a micro simulated moving bed chromatography (μSMB) system, we investigated whether the 2D laminar flow model can be extended to account for additional dispersion effects, such as Dean vortices, through an equivalent radial flow rate distribution (eqFRD). For this purpose, we conducted 3D CFD simulations of the respective ECV and adapted the radial flow rate profile of a 2D simulation to match the residence time distribution observed in the CFD results. Applying the eqFRD model led to a significant improvement in prediction accuracy for isolated ECV segments, increasing from 90% to 97% compared to traditional ECV models. However, when these models were applied to the full μSMB system, the choice of ECV model had minimal impact on overall results as long as retention time within the ECV was accurately predicted. This suggests that, in the studied system, the column has a greater influence on peak shape than the ECV, allowing simpler ECV models to suffice in certain contexts. Despite these advances, significant deviations between predicted and experimental results were observed, indicating that factors such as the transition between the column and ECV, as well as detector effects, should be considered in future research. The results underscore the importance of selecting an ECV model in the context of the entire system, balancing accuracy with computational efficiency.

Statistics and Learning Theory in the Era of Artificial Intelligence

The workshop highlighted recent theoretical advances on inference in high-dimensional statistical models based on the interplay of techniques from mathematical statistics, machine learning, theoretical computer science and related areas. The workshop brought together about 50 researchers in order to present new results, exchange ideas and explore open problems.

Deep neural networks for endemic measles dynamics: Comparative analysis and integration with mechanistic models.

Measles is an important infectious disease system both for its burden on public health and as an opportunity for studying nonlinear spatio-temporal disease dynamics. Traditional mechanistic models often struggle to fully capture the complex nonlinear spatio-temporal dynamics inherent in measles outbreaks. In this paper, we first develop a high-dimensional feed-forward neural network model with spatial features (SFNN) to forecast endemic measles outbreaks and systematically compare its predictive power with that of a classical mechanistic model (TSIR). We illustrate the utility of our model using England and Wales measles data from 1944-1965. These data present multiple modeling challenges due to the interplay between metapopulations, seasonal trends, and nonlinear dynamics related to demographic changes. Our results show that while the TSIR model yields similarly performant short-term (1 to 2 biweeks ahead) forecasts for highly populous cities, our neural network model (SFNN) consistently achieves lower root mean squared error (RMSE) across other forecasting windows. Furthermore, we show that our spatial-feature neural network model, without imposing mechanistic assumptions a priori, can uncover gravity-model-like spatial hierarchy of measles spread in which major cities play an important role in driving regional outbreaks. We then turn our attention to integrative approaches that combine mechanistic and machine learning models. Specifically, we investigate how the TSIR can be utilized to improve a state-of-the-art approach known as Physics-Informed-Neural-Networks (PINN) which explicitly combines compartmental models and neural networks. Our results show that the TSIR can facilitate the reconstruction of latent susceptible dynamics, thereby enhancing both forecasts in terms of mean absolute error (MAE) and parameter inference of measles dynamics within the PINN. In summary, our results show that appropriately designed neural network-based models can outperform traditional mechanistic models for short to long-term forecasts, while simultaneously providing mechanistic interpretability. Our work also provides valuable insights into more effectively integrating machine learning models with mechanistic models to enhance public health responses to measles and similar infectious disease systems.

Contrast and content preserving HDMR-based color-to-gray conversion

Converting a color image into a grayscale image is a complex problem that is based on preserving color contrast, sharpness, and luminance. In this paper, a novel image decolorization algorithm is proposed using High Dimensional Model Representation (HDMR) with an optimization procedure that retains color content and contrast. In the proposed algorithm, a color image is first decomposed into HDMR components and then the components are categorized depending on whether they are colored or colorless. After that, the image is reconstructed by merging the weighted colored and colorless HDMR components. The weight coefficients are determined by an optimization process. The proposed algorithm both visually and quantitatively compared with state-of-the-art methods in the literature using various performance evaluation metrics. As regards all obtained results, the HDMR based image decolorization algorithm is more potent and has better performance in overall comparison. Most importantly, this algorithm has a flexible structure as it is able to produce various grayscale images for different thresholds of visible color contrast which makes this algorithm superior given that it is the only one that accomplishes this feat in the literature to the best of our knowledge.

Cosmological Models in Lovelock Gravity: An Overview of Recent Progress

In the current review, we provide a summary of the recent progress made in the cosmological aspect of extra-dimensional Lovelock gravity. Our review covers a wide variety of particular model/matter source combinations: Einstein–Gauss–Bonnet as well as cubic Lovelock gravities with vacuum, cosmological constant, perfect fluid, spatial curvature, and some of their combinations. Our analysis suggests that it is possible to set constraints on the parameters of the above-mentioned models from the simple requirement of the existence of a smooth transition from the initial singularity to a realistic low-energy regime. Initially, anisotropic space naturally evolves into a configuration with two isotropic subspaces, and if one of these subspaces is three-dimensional and is expanding while another is contracting, we call it realistic compactification. Of course, the process is not devoid of obstacles, and in our paper, we review the results of the compactification occurrence investigation for the above-mentioned models. In particular, for vacuum and Λ-term EGB models, compactification is not suppressed (but is not the only possible outcome either) if the number of extra dimensions is D⩾2; for vacuum cubic Lovelock gravities it is always present (however, cubic Lovelock gravity is defined only for D⩾3 number of extra dimensions); for the EGB model with perfect fluid it is present for D=2 (we have not considered this model in higher dimensions yet), and in the presence of spatial curvature, the realistic stabilization of extra dimensions is always present (however, such a model is well-defined only in D⩾4 number of extra dimensions).

A new variable selection and estimation algorithm for the high-dimensional quantile single-index model

The single-index model (SIM) reveals the intricate relationship between the response variable and covariates, allowing for the consideration of potential heterogeneity and nonlinearity in the data, while also offering interpretability and flexibility. This paper concentrates on estimating the parameters and the unknown link function for the quantile SIM with high-dimensional covariates. We introduce a novel iterative algorithm that strikes a balance between estimation accuracy, computational efficiency, and adaptability to diverse datasets. The initial values for iteration and iteration termination conditions are also discussed. The finite sample performance is illustrated through a simulation study, demonstrating the advantages of our method in terms of accuracy and speed.

Multiobjective Route Optimization for Multimodal Cold Chain Networks Considering Carbon Emissions and Food Waste

The cold chain logistics industry faces significant challenges in terms of transportation costs and carbon emissions. It is imperative to plan multimodal transportation routes efficiently to address these issues, minimize food waste, and reduce carbon emissions. This paper focuses on four key optimization objectives for multimodal cold chain transport: minimizing total transportation time, costs, carbon emissions, and food waste. To tackle these objectives, we propose a high-dimensional multiobjective route optimization model for multimodal cold chain networks. Our approach involves the development of a multiobjective evolutionary algorithm, utilizing Monte Carlo simulation and a one-by-one selection strategy. We evaluate the proposed algorithm’s performance by analyzing various convergence and distribution indicators. The average values for the minimum total transportation time, transportation cost, carbon emission cost, and cargo loss rate derived from the proposed algorithm ultimately converge to 6721.7, 5184.4, 301.5, and 0.21, respectively, demonstrating the effectiveness of the algorithmic solution. Additionally, we benchmark our algorithm against the existing literature to showcase its efficiency in solving high-dimensional multi-objective route optimization problems. Furthermore, we investigate the impact of different parameters, such as carbon tax rates, temperature, and cargo activation energy, on carbon emissions, and food waste. Moreover, we conduct a real-world case study to apply our approach to solving a practical business problem related to multimodal cold chain transportation. The insights gained from this research offer valuable decision-making support for multimodal carriers in developing low-carbon and environmentally friendly transportation strategies to efficiently transport perishable goods.

-Penalized Multinomial Regression: Estimation, Inference, and Prediction, With an Application to Risk Factor Identification for Different Dementia Subtypes.

High-dimensional multinomial regression models are very useful in practice but have received less research attention than logistic regression models, especially from the perspective of statistical inference. In this work, we analyze the estimation and prediction error of the contrast-based -penalized multinomial regression model and extend the debiasing method to the multinomial case, providing a valid confidence interval for each coefficient and value of the individual hypothesis test. We also examine cases of model misspecification and non-identically distributed data to demonstrate the robustness of our method when some assumptions are violated. We apply the debiasing method to identify important predictors in the progression into dementia of different subtypes. Results from extensive simulations show the superiority of the debiasing method compared to other inference methods.

A High dimensional and high sigma statistical model for SRAM read access yield estimation

A High dimensional and high sigma statistical model for SRAM read access yield estimation

Robust Estimation for Number of Factors in High Dimensional Factor Modeling via Spearman Correlation Matrix

Determining the number of factors in high-dimensional factor modeling is essential but challenging, especially when the data are heavy-tailed. In this article, we introduce a new estimator based on the spectral properties of Spearman sample correlation matrix under the high-dimensional setting, where both dimension and sample size tend to infinity proportionally. Our estimator is robust against heavy tails in either the common factors or idiosyncratic errors. The consistency of our estimator is established under mild conditions. Numerical experiments demonstrate the superiority of our estimator compared to existing methods. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Method for Constructing a Compact Component Mode Synthesis Model for Analyzing Nonlinear Normal Modes of Structures with Localized Nonlinearities

Abstract Nonlinear dynamics analysis is a crucial topic in mechanical and aerospace engineering. The analysis of nonlinear normal modes (NNMs) provides an effective mathematical tool for interpreting complex nonlinear vibration phenomena. Unlike the invariant normal modes of linear systems, NNMs often exhibit frequency-energy dependence and cannot be computed using the traditional eigen-decomposition method. Calculating NNMs relies on numerical methods that involve expensive iterative computations, especially for systems with numerous degrees-of-freedom. To relieve computational costs, this paper proposes a mode selection method for the component mode synthesis (CMS) technique to enable compact reduced-order modeling of structures with localized nonlinearities. The reduced-order model is then combined with a numerical continuation scheme to establish a low-cost NNM analysis framework. This framework can efficiently predict the NNMs of high-dimensional finite element models. The proposed framework is demonstrated on an I-shaped cantilever beam with localized nonlinear stiffness. The results show that the proposed approach can identify the key modes in the CMS modeling procedure. The NNMs of the nonlinear I-shaped beam structure can then be analyzed with a significant reduction in computational costs.

A comprehensive estimation method for vehicle motion states based on model constraints

The motion state estimation is indispensable for unmanned ground vehicles. Most traditional methods estimate individual states and utilize high-dimensional dynamics models that necessitate extensive prior information to improve accuracy. However, these approaches are beset with numerous challenges in the accuracy, complexity, and computational burden. In this paper, a novel method is proposed to address these problems with two novelties. Firstly, it possesses the capability to simultaneously estimate 10 fundamental states, including vehicle velocities, wheel slip ratios, and wheel sideslip velocities. The estimation accuracy is augmented since the coupling relationships among states are considered. Secondly, the method avoids the utilization of the complete vehicle model, instead employing the constraints derived from the models, thereby the requisite prior information and computational burden can be reduced. The accuracy of the proposed method was demonstrated through simulation experiments and field tests, as well as its robustness in challenging situations, such as the wheel slip.

Stationary Distribution and Density Function for a High-Dimensional Stochastic SIS Epidemic Model with Mean-Reverting Stochastic Process

This paper explores a high-dimensional stochastic SIS epidemic model characterized by a mean-reverting, stochastic process. Firstly, we establish the existence and uniqueness of a global solution to the stochastic system. Additionally, by constructing a series of appropriate Lyapunov functions, we confirm the presence of a stationary distribution of the solution under R0s>1. Taking 3D as an example, we analyze the local stability of the endemic equilibrium in the stochastic SIS epidemic model. We introduce a quasi-endemic equilibrium associated with the endemic equilibrium of the deterministic system. The exact probability density function around the quasi-stable equilibrium is determined by solving the corresponding Fokker–Planck equation. Finally, we conduct several numerical simulations and parameter analyses to demonstrate the theoretical findings and elucidate the impact of stochastic perturbations on disease transmission.