High-dimensional Dependence Research Articles

Learning the Graph Structure of Regular Vine-Copulas from Dependence Lists

Regular vine copulas (R-vines) provide a comprehensive framework for modeling high-dimensional dependencies using a hierarchy of trees and conditional pair-copulas. While the graphical structure of R-vines is traditionally derived from data, this work introduces a novel approach by utilizing a (conditional) pairwise dependence list. Our primary goal is to construct R-vine graphs that include the maximum possible number of dependence relationships specified in such lists. To tackle this optimization challenge, characterized by exponential growth in the search space and the structural constraints of R-vines, we propose two distinct methodologies: A 0-1 linear programming formulation and a Genetic Algorithm (GA). Additionally, the Randomized Constructive Technique (RCT) is employed to generate the initial population of the GA, serving as a baseline for our comparison. Experimental results reveal the superior performance of the GA over the RCT in terms of success rate, incorporating more relationships than RCT into the constructed R-vine graphs and achieving near-optimal or optimal graph structures.

Vine copulas for accelerated prediction of composite strength variability

Composite materials are essential for many advanced engineering applications, but numerical quantification of strength variability can be computationally expensive. This paper proposes a novel methodology that drastically reduces the number of finite element simulations required to characterise composite material strength variability using the concept of vine copulas. This concept provides a flexible tool for the representation of high-dimensional dependencies. The methodology considers spatial scatter of three material variabilities: fibre volume fraction, fibre misalignment and fibre strength. First, the cross-correlation between stress, fibre volume fraction and fibre misalignment is fitted by a vine copula using results from a limited number of finite element simulations. Next, the vine copula is used to predict new conditional stress realisations when given realisations for the fibre volume fraction and fibre misalignment. This effectively replaces the finite element simulations with a vine copula that is much faster to evaluate. The methodology is verified by predicting the tensile failure load of a unidirectional composite coupon. Predictions are very similar to an exclusively finite element-based approach, while reducing the number of finite element simulations by a factor of 200.

An efficient cross-entropy method addressing high-dimensional dependencies for composite systems reliability evaluation

Cross-entropy (CE) based importance sampling (IS) accelerates the reliability evaluation of power system greatly, but is mainly focused on the CEIS of independent random variables (RVs) or low-dimensional correlated RVs (CRVs). To extend the CE method to high-dimensional CRVs while avoiding the “curse of dimensionality” in optimizing a high-dimensional IS probability density function (IS-PDF), an efficient CE method termed as CE-DRDM, is developed from a dimension-reduced dependence model (DRDM). First, based on the DRDM’s original hierarchical disaggregate structure (HDS), the CE-DRDM which conducts the CEIS for a single aggregate RV rather than the high-dimensional CRVs is proposed. Second, to solve the issue that the performance of CE-DRDM may degrade in the case of high penetration of renewable energy, the CE-DRDM is enhanced by improving the DRDM’s HDS, and then a novel CE optimization method is proposed, through which the analytical updating formulas of IS-PDF parameters can be derived. Thirdly, the CE-DRDM is further enhanced by using a sparse density estimator, based on which the number of IS-PDF parameters needed to be optimized in the CE optimization reduces greatly, and consequently the optimization accuracy improves accordingly. Finally, the validity of CE-DRDM is verified by several numerical cases with high-dimensional dependencies.

Impact of diverse configuration in multivariate bias correction methods on large-scale hydrological modelling under climate change

For climate change impact studies, bias correction methods are widely utilized to improve climate model outputs by correcting the representation of climate variables. In addition to the univariate bias correction methods, many multivariate bias correction methods have been proposed, using diverse mathematical concepts in their individual designs. Although several attempts have been made to evaluate the performance of the multivariate bias correction methods, there is no exploration in the impact of diverse configurations to target multivariate dependencies in the multivariate bias correction methods particularly for hydrological modeling. This study tackles this research gap by developing an inter-comparison framework that assesses how the different configurations affect the climate and hydrological simulations. To achieve the research goal, the impact of three configurations (inter-variable, inter-spatial, and full-dimensional dependence configurations) is evaluated on multiple basins in a national-scale domain over South Korea. Results show that although most methods in the three different configurations capably correct the statistics, some instabilities are also observed when a high-dimensional dependence configuration is used, informing that some multivariate bias correction methods may not be compatible with high-dimensional configurations. For hydrological modeling, the configuration definitions bring about distinct differences in streamflow realizations. Only some of the inter-variable and full-dimensional configurations perform better or similarly to univariate bias correction methods. Finally, we demonstrate the importance of selecting bias correction methods while addressing the necessity for further improvement.

Extreme risk spillovers between China and major international stock markets

We examine the complex dependence structure and risk spillovers between the Chinese stock market and twelve major international markets. To this end, we employ three types of vine copulas and tests for the Granger causality in risk of Hong et al. (2009). The results indicate that the R-vine copula is the optimal model to characterize the high-dimensional dependence structure of the markets after China joined the WTO, which suggests obvious structural differences with varying degrees of mainly positive dependences. Moreover, we identify unilateral extreme risk spillovers from China to the United States, France, and Germany, and either from Japan to China. We also detect bilateral spillovers between China and the United States, Japan, as well as Australia.

Generating multivariate load states using a conditional variational autoencoder

For planning of power systems and for the calibration of operational tools, it is essential to analyse system performance in a large range of representative scenarios. When the available historical data is limited, generative models are a promising solution, but modelling high-dimensional dependencies is challenging. In this paper, a multivariate load state generating model on the basis of a conditional variational autoencoder (CVAE) neural network is proposed. Going beyond common CVAE implementations, the model includes stochastic variation of output samples under given latent vectors and co-optimizes the parameters for this output variability. It is shown that this improves statistical properties of the generated data. The quality of generated multivariate loads is evaluated using univariate and multivariate performance metrics. A generation adequacy case study on the European network is used to illustrate model's ability to generate realistic tail distributions. The experiments demonstrate that the proposed generator outperforms other data generating mechanisms.

Modelling the galaxy–halo connection with machine learning

ABSTRACT To extract information from the clustering of galaxies on non-linear scales, we need to model the connection between galaxies and haloes accurately and in a flexible manner. Standard halo occupation distribution (HOD) models make the assumption that the galaxy occupation in a halo is a function of only its mass, however, in reality; the occupation can depend on various other parameters including halo concentration, assembly history, environment, and spin. Using the IllustrisTNG hydrodynamical simulation as our target, we show that machine learning tools can be used to capture this high-dimensional dependence and provide more accurate galaxy occupation models. Specifically, we use a random forest regressor to identify which secondary halo parameters best model the galaxy–halo connection and symbolic regression to augment the standard HOD model with simple equations capturing the dependence on those parameters, namely the local environmental overdensity and shear, at the location of a halo. This not only provides insights into the galaxy formation relationship but also, more importantly, improves the clustering statistics of the modelled galaxies significantly. Our approach demonstrates that machine learning tools can help us better understand and model the galaxy–halo connection, and are therefore useful for galaxy formation and cosmology studies from upcoming galaxy surveys.

Neural Network Modeling of NiTiHf Shape Memory Alloy Transformation Temperatures

Data-driven techniques are used to predict the transformation temperatures (TTs) of NiTiHf shape memory alloy. A machine learning (ML) approach is used to overcome the high-dimensional dependency of NiTiHf TTs on numerous factors, as well as the lack of fully known governing physics. The elemental composition, thermal treatments, and post-processing steps that are commonly used to process NiTiHf and have an impact on the material phase transitions are used as input parameters of the neural network model (NN) to design the TTs. Such a feature selection led to the use of most of the accessible information in the literature on NiTiHf TTs, as all processing features required to be fed into the NN model. Considering most of the regular NiTiHf processing factors also enables the option of tuning additional characteristics of NiTiHf in addition to the TTs. The work is unique as all the four main TTs and their associated peak transformation temperatures are predicted to have complete control over the material phase change thresholds. Since 1995, extensive experimental research has been conducted to design NiTiHf TTs with a large temperature range of around 800 °C, paving the path for the current work’s ML algorithms to be fed. A thorough data collection is created using both unpublished data and available literature and then analyzed to select twenty input parameters to feed the NN model. To forecast the NiTiHf TTs, a total of 173 data points were gathered, verified, and selected. The model's overall determination factor (R2) was 0.96, suggesting the viability of the proposed NN model in demonstrating the link between material composition and processing factors, as well as identifying the TTs of NiTiHf alloy. The effort additionally validates the generated results against existing data in the literature. The validation confirms the significance of the proposed model.

Parametric Copula-GP model for analyzing multidimensional neuronal and behavioral relationships.

One of the main goals of current systems neuroscience is to understand how neuronal populations integrate sensory information to inform behavior. However, estimating stimulus or behavioral information that is encoded in high-dimensional neuronal populations is challenging. We propose a method based on parametric copulas which allows modeling joint distributions of neuronal and behavioral variables characterized by different statistics and timescales. To account for temporal or spatial changes in dependencies between variables, we model varying copula parameters by means of Gaussian Processes (GP). We validate the resulting Copula-GP framework on synthetic data and on neuronal and behavioral recordings obtained in awake mice. We show that the use of a parametric description of the high-dimensional dependence structure in our method provides better accuracy in mutual information estimation in higher dimensions compared to other non-parametric methods. Moreover, by quantifying the redundancy between neuronal and behavioral variables, our model exposed the location of the reward zone in an unsupervised manner (i.e., without using any explicit cues about the task structure). These results demonstrate that the Copula-GP framework is particularly useful for the analysis of complex multidimensional relationships between neuronal, sensory and behavioral variables.

Electricity‐heat‐gas integrated demand response dependency assessment based on BOXCOX‐Pair Copula model

With the continuous development of Regional Integrated Energy System (RIES), demand response (DR) is composed of diversified loads including electric load, heat load and gas load. Their cross-dependencies reflecting the nonlinear coupling complementary relationship between each load type are one of the key factors to improve multi-energy flow optimisation modelling and utilisation efficiency on the demand side. Accordingly, this paper proposes a DR dependency assessment method considering electricity-heat-gas loads based on the BOXCOX-Pair Copula model. BOXCOX transformation is introduced to convert various probability distribution statistics of electricity-heat-gas loads into Gaussian distributed variables. C-vine Pair Copula is used to characterise an ensemble-of-trees of high-dimensional dependency structure among multi-energy demand modalities. Then the combined model of BOXCOX transformation and C-vine Pair Copula can be employed to determine the complex coupling dependency among electricity-heat-gas loads according to different DR statistics. Some metrics between the original empirical distribution and BOXCOX-Pair Copula distribution are introduced to assess the dependency evaluation precision of the proposed model. Finally, the novel dependency assessment model is numerically tested utilising the electricity load, heat load and gas load data sequences in a real RIES. The results illustrate that the cross-dependency of electricity-heat-gas integrated DR based on the BOXCOX-C-vine copula model is closer to that of actual sample data, which verify the effectiveness and superiority of the proposed approach.

Flexibility-constraint integrated resource planning framework considering demand and supply side uncertainties with high dimensional dependencies

The propensity to utilization of inexpensive, clean and distributed energy resources in the today's power systems structure are expanding due to various economic, political and sociological reasons. However, the power generated by these resources is extremely unpredictable, which is called uncertainty and exceptionally fluctuated drastically that is known as variability, during the operation epoch. So, exposure to these two features is great significance and value. This problem is known as the power system flexibility in the literature of electrical power system studies. Promoting flexibility in the system is too expensive and can be evaluated in different time frames. This paper studies the power system flexibility with the presence of uncertainty and variability of demand and supply side resources in a planning timeframe. This paper proposed an improved method for demand response uncertainty modeling with Z-number. This method is a hybrid possibilistic and probabilistic process. Furthermore, this paper applied an approved method for renewable generation uncertainty modeling with Monte Carlo. The scrutinies show, the most cost-effective way to consider the flexibility is planning timeframe, so this study proposed a novel paradigm for Flexibility-Constrained Integrated resource planning in presence of high penetration uncertain and variable demand-side and supply-side resources namely as FCIRPU/V-DSRs. The simulations are implemented with the proposed FCIRPU/V-DSRs methodology on a test system. The results show that this innovative framework provides a significant improvement in the power system flexibility at an inferior cost than other investigated methods.

Multi-factor dependence modelling with specified marginals and structured association in large-scale project risk assessment

This paper examines the high-dimensional dependence modelling problem in the context of project risk assessment. As the dimension of uncertain performance units (i.e., itemized costs and activity times) in a project increases, specifying a feasible correlation matrix and eliciting relevant pair-wise information, either from historical data or with expert judgement, becomes practically unattainable or simply not economical. This paper presents a factor-driven dependence elicitation and modelling framework with scalability to large-scale project risks. The multi-factor association model (MFAM) accounts for hierarchical relationships of multiple association factors and provides a closed-form solution to a complete and mathematically consistent correlation matrix. Augmented with the structured association (SA) technique for systematic identification of hierarchical association factors, the MFAM offers additional flexibility of utilizing the minimum information available in standardized, ubiquitous project plans (e.g., work breakdown structure, resource allocation, or risk register), while preserving the computational efficiency and the scalability to high dimensional project risks. Numerical applications and simulation experiments show that the MFAM, further combined with extended analytics (i.e., parameter calibration and optimization), provides credible risk assessments (with accuracy comparable to full-scale simulation) and further enhances the realism of dealing with high-dimensional project risks utilizing all relevant information.

Modelling mortality dependence: An application of dynamic vine copula

Vine copula, constructed from bivariate copulas, provides great flexibility in modelling complex high-dimensional dependence. When applied to multi-population mortality modelling, vine copula yields significant improvement over traditional multivariate copulas. In this paper, we propose to capture time-varying features in mortality dependence with dynamic regular vine (R-vine) copula which is built from bivariate copulas with time-varying dependence parameters. We develop two dependence dynamics for R-vine copulas and illustrate the selection and estimation of dynamic R-vine copulas using mortality data from eight populations. The estimated R-vine copulas using the proposed dependence dynamics are shown to yield better goodness of fit than both static and regime-switching vine copulas. We further demonstrate the simulation of mortality paths using dynamic R-vine copulas and examine the impact of vine copula choice on the assessed effectiveness of longevity hedge.

Dependence Modeling for Large-scale Project Cost and Time Risk Assessment: Additive Risk Factor Approaches

High-dimensional dependence modeling remains a crucial challenge in quantitative project cost and time risk analysis. Building a complete and mathematically consistent correlation matrix becomes unrealistically restrictive as the number of uncertain performance units in a project (i.e., activity times and costs) increases, regardless of using empirical data or with subjective judgment. This article presents a pair of additive factor dependence models that provide analytic solutions to the generation of a complete and mathematically consistent correlation matrix. The additive risk factor (ARF) models account for multiple risk factors in two classes (i.e., extra-marginal and intramarginal) while providing additional flexibility for a strategic tradeoff between the accuracy and the scalability to high-dimensional project risks. We extend the ARF models to present an analytic solution to the program evaluation and review technique (PERT) problem with correlated activity times. Numerical examples demonstrate the accuracy and computational efficiency of the ARF approaches. The ARF approaches and the ARF-PERT would serve as a quick and sensible alternative to large-scale Monte Carlo simulation, in particular during the early stage of the project life-cycle.

Modeling high-dimensional dependence in astronomical data

Fixing the relationship of a set of experimental quantities is a fundamental issue in many scientific disciplines. In the 2D case, the classical approach is to compute the linear correlation coefficient ρ from a scatterplot. This method, however, implicitly assumes a linear relationship between the variables. Such an assumption is not always correct. With the use of the partial correlation coefficients, an extension to the multidimensional case is possible. However, the problem of the assumed mutual linear relationship of the variables remains. A relatively recent approach that makes it possible to avoid this problem is the modeling of the joint probability density function of the data with copulas. These are functions that contain all the information on the relationship between two random variables. Although in principle this approach also can work with multidimensional data, theoretical as well computational difficulties often limit its use to the 2D case. In this paper, we consider an approach based on so-called vine copulas, which overcomes this limitation and at the same time is amenable to a theoretical treatment and feasible from the computational point of view. We applied this method to published data on the near-IR and far-IR luminosities and atomic and molecular masses of the Herschel reference sample, a volume-limited sample in the nearby Universe. We determined the relationship of the luminosities and gas masses and show that the far-IR luminosity can be considered as the key parameter relating the other three quantities. Once removed from the 4D relation, the residual relation among the latter is negligible. This may be interpreted as the correlation between the gas masses and near-IR luminosity being driven by the far-IR luminosity, likely by the star formation activity of the galaxy.

Leverage, Asymmetry, and Heavy Tails in the High-Dimensional Factor Stochastic Volatility Model

We develop a factor stochastic volatility model that incorporates leverage effects, return asymmetry, and heavy tails across all systematic and idiosyncratic model components. Our model leads to a flexible high-dimensional dependence structure that allows for time-varying correlations, tail dependence, and volatility response to both systematic and idiosyncratic return shocks. We develop an efficient Markov chain Monte Carlo algorithm for posterior estimation based on the particle Gibbs, ancestor sampling, particle efficient importance sampling methods, and interweaving strategy. To obtain parsimonious specifications in practice, we build computationally efficient model selection directly into our estimation algorithm. We validate the performance of our proposed estimation method via simulation studies with different model specifications. An empirical study for a sample of U.S. stocks shows that return asymmetry is a systematic phenomenon and our model outperforms other factor models for value-at-risk evaluation.

Modeling of Stochastic Wind Based on Operational Flight Data Using Karhunen-Loève Expansion Method.

Wind has a significant influence on the operational flight safety. To quantify the influence of the wind characteristics, a wind series generator is required in simulations. This paper presents a method to model the stochastic wind based on operational flight data using the Karhunen–Loève expansion. The proposed wind model allows us to generate new realizations of wind series, which follow the original statistical characteristics. To improve the accuracy of this wind model, a vine copula is used in this paper to capture the high dimensional dependence among the random variables in the expansions. Besides, the proposed stochastic model based on the Karhunen–Loève expansion is compared with the well-known von Karman turbulence model based on the spectral representation in this paper. Modeling results of turbulence data validate that the Karhunen–Loève expansion and the spectral representation coincide in the stationary process. Furthermore, construction results of the non-stationary wind process from operational flights show that the generated wind series have a good match in the statistical characteristics with the raw data. The proposed stochastic wind model allows us to integrate the new wind series into the Monte Carlo Simulation for quantitative assessments.

Detection of Local Differences in Spatial Characteristics Between Two Spatiotemporal Random Fields

Comparing the spatial characteristics of spatiotemporal random fields is often at demand. However, the comparison can be challenging due to the high-dimensional feature and dependency in the data. We develop a new multiple testing approach to detect local differences in the spatial characteristics of two spatiotemporal random fields by taking the spatial information into account. Our method adopts a two-component mixture model for location wise p-values and then derives a new false discovery rate (FDR) control, called mirror procedure, to determine the optimal rejection region. This procedure is robust to model misspecification and allows for weak dependency among hypotheses. To integrate the spatial heterogeneity, we model the mixture probability as well as study the benefit if any of allowing the alternative distribution to be spatially varying. An EM-algorithm is developed to estimate the mixture model and implement the FDR procedure. We study the FDR control and the power of our new approach both theoretically and numerically, and apply the approach to compare the mean and teleconnection pattern between two synthetic climate fields. Supplementary materials for this article are available online.

Dimension Reduction Based Non-Parametric Disaggregation for Dependence Modeling in Composite System Reliability Evaluation

The correlated variation of bus loads and wind powers has posed a challenge to power system security. How to construct an accurate dependence model becomes vital for power system reliability evaluation. Most currently used methods in estimating the multivariate probability density function (PDF) not only ignore the aggregation constraint but also encounter the curse of dimensionality. In this paper, a new non-parametric disaggregation (ND) technique is presented to disaggregate an aggregate variable (e.g. system load) into non-aggregate variables (e.g. bus loads) while meeting the aggregation constraint. An innovative hierarchical dimension reduction technique is proposed and integrated into the ND model to transform a high-dimensional conditional PDF estimation into some low-dimensional estimations, which effectively resolves the difficulty associated with high dimensionality. The above two techniques are incorporated into an effective three-stage conditional sampling method to form the proposed dimension reduction based ND model (DRND) for high-dimensional dependence modeling in composite system reliability evaluation. The validity of the proposed DRND is verified by three test systems, i.e. MRTS79, MRTS96, and the latest RTS-GMLC.

Improved probabilistic load flow method based on D‐vine copulas and Latin hypercube sampling in distribution network with multiple wind generators

In this study, a D-vine copulas modelling based probabilistic load flow (PLF) computation method is proposed, which considers the dependence among multiple wind generators. Furthermore, this method is not restricted by the type of wind speed distribution, i.e. allow random variables to comply with any types of distribution model. Copula theory plays an important role on dependency modelling. However, when high-dimensional correlation is taken into account, standard multivariate copula suffers from the problems of inflexible structure. Vine copula is flexible to build high-dimensional dependence and able to construct complicated dependence structure by applying bivariate copulas. For marginal distributions of wind speed, non-parametric model can provide a better estimation than those parametric models. An improved Latin hypercube sampling based Monte Carlo simulation method is utilised to solve PLF problems. A modified IEEE 33-node distribution system is used to conduct the numerical experiments for the accuracy and efficiency verification of the proposed PLF method, under the MatlabR2016a platform. The simulation results verify the outstanding accuracy, efficiency and robustness of the proposed PLF method.