Complex Dependence Structures Research Articles

A Data-Driven Probabilistic Network Approach to Assess Model Similarity in CMIP Ensembles

Abstract The different phases of the Coupled Model Intercomparison Project (CMIP) provide ensembles of past, present, and future climate simulations crucial for climate change impact and adaptation activities. These ensembles are produced using multiple global climate models (GCMs) from different modeling centers with some shared building blocks and interdependencies. Applications typically follow the “model democracy” approach which might have significant implications in the resulting products (e.g., large bias and low spread). Thus, quantifying model similarity within ensembles is crucial for interpreting model agreement and multimodel uncertainty in climate change studies. The classical methods used for assessing GCM similarity can be classified into two groups. The a priori approach relies on expert knowledge about the components of these models, while the a posteriori approach seeks similarity in the GCMs’ output variables and is thus data-driven. In this study, we apply probabilistic network models (PNMs), a well-established machine learning technique, as a new a posteriori method to measure intermodel similarities. The proposed methodology is applied to surface temperature fields of the historical experiments from the CMIP5 multimodel ensemble and different reanalysis gridded datasets. PNMs are able to learn the complex spatial dependency structures present in climate data, including teleconnections operating on multiple spatial scales, characteristic of the underlying GCM. A distance metric building on the resulting PNMs is applied to characterize GCM model dependencies. The results of this approach are in line with those obtained with more traditional methods but have further explanatory potential building on probabilistic model querying. Significance Statement The present study proposes the use of probabilistic network models (PNMs) to quantify model similarity within ensembles of global climate models (GCMs). This is crucial for interpreting model agreement and multimodel uncertainty in climate change studies. When applied to climate data (gridded global surface temperature in this study), PNMs encode the relevant spatial dependencies (local and remote connections). Similarities among the PNMs resulting from different GCMs can be quantified and are shown to capture similar GCM formulations reported in previous studies. Differently to other machine learning methods previously applied to this problem, PNMs are fully explainable (allowing probabilistic querying) and are applicable to high-dimensional gridded raw data.

Bayesian inference for identifying tumour-specific cancer dependencies through integration of ex-vivo drug response assays and drug-protein profiling

The identification of tumor-specific molecular dependencies is essential for the development of effective cancer therapies. Genetic and chemical perturbations are powerful tools for discovering these dependencies. Even though chemical perturbations can be applied to primary cancer samples at large scale, the interpretation of experiment outcomes is often complicated by the fact that one chemical compound can affect multiple proteins. To overcome this challenge, Batzilla et al. (PLoS Comput Biol 18(8): e1010438, 2022) proposed DepInfeR, a regularized multi-response regression model designed to identify and estimate specific molecular dependencies of individual cancers from their ex-vivo drug sensitivity profiles. Inspired by their work, we propose a Bayesian extension to DepInfeR. Our proposed approach offers several advantages over DepInfeR, including e.g. the ability to handle missing values in both protein-drug affinity and drug sensitivity profiles without the need for data pre-processing steps such as imputation. Moreover, our approach uses Gaussian Processes to capture more complex molecular dependency structures, and provides probabilistic statements about whether a protein in the protein-drug affinity profiles is informative to the drug sensitivity profiles. Simulation studies demonstrate that our proposed approach achieves better prediction accuracy, and is able to discover unreported dependency structures.

Estimation of multiple networks with common structures in heterogeneous subgroups

Network estimation has been a critical component of high-dimensional data analysis and can provide an understanding of the underlying complex dependence structures. Among the existing studies, Gaussian graphical models have been highly popular. However, they still have limitations due to the homogeneous distribution assumption and the fact that they are only applicable to small-scale data. For example, cancers have various levels of unknown heterogeneity, and biological networks, which include thousands of molecular components, often differ across subgroups while also sharing some commonalities. In this article, we propose a new joint estimation approach for multiple networks with unknown sample heterogeneity, by decomposing the Gaussian graphical model (GGM) into a collection of sparse regression problems. A reparameterization technique and a composite minimax concave penalty are introduced to effectively accommodate the specific and common information across the networks of multiple subgroups, making the proposed estimator significantly advancing from the existing heterogeneity network analysis based on the regularized likelihood of GGM directly and enjoying scale-invariant, tuning-insensitive, and optimization convexity properties. The proposed analysis can be effectively realized using parallel computing. The estimation and selection consistency properties are rigorously established. The proposed approach allows the theoretical studies to focus on independent network estimation only and has the significant advantage of being both theoretically and computationally applicable to large-scale data. Extensive numerical experiments with simulated data and the TCGA breast cancer data demonstrate the prominent performance of the proposed approach in both subgroup and network identifications.

High-dimensional sparse vine copula regression with application to genomic prediction.

High-dimensional data sets are often available in genome-enabled predictions. Such data sets include nonlinear relationships with complex dependence structures. For such situations, vine copula-based (quantile) regression is an important tool. However, the current vine copula-based regression approaches do not scale up to high and ultra-high dimensions. To perform high-dimensional sparse vine copula-based regression, we propose 2 methods. First, we show their superiority regarding computational complexity over the existing methods. Second, we define relevant, irrelevant, and redundant explanatory variables for quantile regression. Then, we show our method's power in selecting relevant variables and prediction accuracy in high-dimensional sparse data sets via simulation studies. Next, we apply the proposed methods to the high-dimensional real data, aiming at the genomic prediction of maize traits. Some data processing and feature extraction steps for the real data are further discussed. Finally, we show the advantage of our methods over linear models and quantile regression forests in simulation studies and real data applications.

Disentanglement with Futures, State, and Interaction

Recent work has proposed a memory property for parallel programs, called disentanglement, and showed that it is pervasive in a variety of programs, written in different languages, ranging from C/C++ to Parallel ML, and showed that it can be exploited to improve the performance of parallel functional programs. All existing work on disentanglement, however, considers the "fork/join" model for parallelism and does not apply to "futures", the more powerful approach to parallelism. This is not surprising: fork/join parallel programs exhibit a reasonably strict dependency structure (e.g., series-parallel DAGs), which disentanglement exploits. In contrast, with futures, parallel computations become first-class values of the language, and thus can be created, and passed between functions calls or stored in memory, just like other ordinary values, resulting in complex dependency structures, especially in the presence of mutable state. For example, parallel programs with futures can have deadlocks, which is impossible with fork-join parallelism. In this paper, we are interested in the theoretical question of whether disentanglement may be extended beyond fork/join parallelism, and specifically to futures. We consider a functional language with futures, Input/Output (I/O), and mutable state (references) and show that a broad range of programs written in this language are disentangled. We start by formalizing disentanglement for futures and proving that purely functional programs written in this language are disentangled. We then generalize this result in three directions. First, we consider state (effects) and prove that stateful programs are disentangled if they are race free. Second, we show that race freedom is sufficient but not a necessary condition and non-deterministic programs, e.g. those that use atomic read-modify-operations and some non-deterministic combinators, may also be disentangled. Third, we prove that disentangled task-parallel programs written with futures are free of deadlocks, which arise due to interactions between state and the rich dependencies that can be expressed with futures. Taken together, these results show that disentanglement generalizes to parallel programs with futures and, thus, the benefits of disentanglement may go well beyond fork-join parallelism.

Vine Copula Model: Application to Chemical Elements in Water Samples

Copula can link the bivariate distribution function with marginal distribution functions without requiring specific information about the interdependence among random variables. There are several types of copulas, such as elliptical copulas, Archimedean copulas, and extreme value copulas. However, in multivariate modeling, each type of copula has limitations in modeling complex dependence structures in terms of symmetry and tail dependence properties. The class of vine copulas overcomes these limitations by constructing multivariate models using bivariate copulas in a tree-like structure. The bivariate copulas used in this study include the Clayton, Gumbel, Frank, Gaussian, and Student's t copula families. This study discusses the construction of vine copula models, parameter estimation, and their applications. The construction of vine copulas is done through the decomposition of conditional probability density functions and substituting bivariate copula density functions into the decomposition results. The data used in the study is the logarithm of the concentration of chemical elements in water samples in Colorado. The parameter estimation method used is pseudo-maximum likelihood with sequential estimation. Model selection is then performed using the Akaike information criterion (AIC) to determine the most suitable model. The results indicate that Caesium and Titanium have a dependency relationship with Scandium. Moreover, Scandium and Titanium exhibit the strongest dependence compared to other variable pairs.

Flexible Lévy-Based Models for Time Series of Count Data with Zero-Inflation, Overdispersion, and Heavy Tails

The explosion of time series count data with diverse characteristics and features in recent years has led to a proliferation of new analysis models and methods. Significant efforts have been devoted to achieving flexibility capable of handling complex dependence structures, capturing multiple distributional characteristics simultaneously, and addressing nonstationary patterns such as trends, seasonality, or change points. However, it remains a challenge when considering them in the context of long-range dependence. The Lévy-based modeling framework offers a promising tool to meet the requirements of modern data analysis. It enables the modeling of both short-range and long-range serial correlation structures by selecting the kernel set accordingly and accommodates various marginal distributions within the class of infinitely divisible laws. We propose an extension of the basic stationary framework to capture additional marginal properties, such as heavy-tailedness, in both short-term and long-term dependencies, as well as overdispersion and zero inflation in simultaneous modeling. Statistical inference is based on composite pairwise likelihood. The model’s flexibility is illustrated through applications to rainfall data in Guinea from 2008 to 2023, and the number of NSF funding awarded to academic institutions. The proposed model demonstrates remarkable flexibility and versatility, capable of simultaneously capturing overdispersion, zero inflation, and heavy-tailedness in count time series data.

Joint modelling of the body and tail of bivariate data

In situations where both extreme and non-extreme data are of interest, modelling the whole data set accurately is important. In a univariate framework, modelling the bulk and tail of a distribution has been extensively studied before. However, when more than one variable is of concern, models that aim specifically at capturing both regions correctly are scarce in the literature. A dependence model that blends two copulas with different characteristics over the whole range of the data support is proposed. One copula is tailored to the bulk and the other to the tail, with a dynamic weighting function employed to transition smoothly between them. Tail dependence properties are investigated numerically and simulation is used to confirm that the blended model is sufficiently flexible to capture a wide variety of structures. The model is applied to study the dependence between temperature and ozone concentration at two sites in the UK and compared with a single copula fit. The proposed model provides a better, more flexible, fit to the data, and is also capable of capturing complex dependence structures.

Extreme risk spillovers between China and major international stock markets

We examine the complex dependence structure and risk spillovers between the Chinese stock market and twelve major international markets. To this end, we employ three types of vine copulas and tests for the Granger causality in risk of Hong et al. (2009). The results indicate that the R-vine copula is the optimal model to characterize the high-dimensional dependence structure of the markets after China joined the WTO, which suggests obvious structural differences with varying degrees of mainly positive dependences. Moreover, we identify unilateral extreme risk spillovers from China to the United States, France, and Germany, and either from Japan to China. We also detect bilateral spillovers between China and the United States, Japan, as well as Australia.

Ensemble learning for score likelihood ratios under the common source problem

AbstractMachine learning‐based score likelihood ratios (SLRs) have emerged as alternatives to traditional likelihood ratios and Bayes factors to quantify the value of evidence when contrasting two opposing propositions. When developing a conventional statistical model is infeasible, machine learning can be used to construct a (dis)similarity score for complex data and estimate the ratio of the conditional distributions of the scores. Under the common source problem, the opposing propositions address if two items come from the same source. To develop their SLRs, practitioners create datasets using pairwise comparisons from a background population sample. These comparisons result in a complex dependence structure that violates the independence assumption made by many popular methods. We propose a resampling step to remedy this lack of independence and an ensemble approach to enhance the performance of SLR systems. First, we introduce a source‐aware resampling plan to construct datasets where the independence assumption is met. Using these newly created sets, we train multiple base SLRs and aggregate their outputs into a final value of evidence. Our experimental results show that this ensemble SLR can outperform a traditional SLR approach in terms of the rate of misleading evidence and discriminatory power and present more consistent results.

The Stability of Islamic Cryptocurrencies and Copula-Based Dependence with Alternative Crypto and Fiat Currencies

Purpose — This study aims to examine Islamic cryptocurrencies and their dependency on foreign exchange markets in vine copula architecture (CD-Vine) and provide a framework for detecting complex dependence structures, risk management implications, and hedging effectiveness. Design/Methodology/Approach — This study used gold-backed cryptocurrencies and three fiat currencies. The vine copula approach was preferred because it applies several distributions and estimates complex dependencies. Hedging effectiveness was measured by constructing simulation-based portfolios optimised with DCC-t-Copula. Benford’s law and realized variance were used to determine the stability of Islamic cryptocurrencies. Findings — According to C-Vine and D-Vine copula models, paper money has a weak tail dependence with gold-backed cryptocurrencies. Only OneGram coin, whose volatility matched the risk of Bitcoin, showed zero irregularities in volume trading. The findings were robust to different estimations based on Minimum Spanning Tree and Dendrogram. Originality/Value — This is the first study to examine Islamic cryptocurrencies’ stability and the significance of hedging effectiveness on gold-backed cryptocurrencies under a copula-based approach. Research Limitations — The study did not apply time-varying vine copula. Practical Implications — The risk management perspective shows insignificant hedge effectiveness in the portfolio of fiat and gold-backed cryptocurrencies.

Vine copula-based Bayesian classification for multivariate time series of electroencephalography eye states

Abstract Sometimes classification tasks have to be based on multivariate time series data collected for each class. In these situations the data for each class might exhibit non-stationary behaviour together with complex dependence structures. We propose a vine copula-based approach to capture these features in each class before applying a Bayesian classifier. Vine copulas have been very successful in modelling asymmetric tail dependence among variables and are coupled with non-stationary univariate time series to model the multivariate time series data for each class. We illustrate this classification approach using data from a neural activity experiment using electroencephalography, where we want to classify the eye state. The level of neural activity was collected over time for multiple locations on the scalp. Our approach is able to identify relevant locations and allows for a model-based interpretation of the data generating process. A cross-validation study with comparison to competitor classifiers for this data set shows good performance of the proposed classifier.

Interference effects on the processing of Korean double relative clauses

The current investigates the parsing process and its utilization of various linguistic information to resolve dependencies in Korean double relative clauses. While previous research has revealed evidence of interference effects of a distractor during dependency formations, a distractor involving dependency formations by itself has not been thoroughly explored. In light of this, we aimed to explore how the grammatical-role parallelism of a distractor modulates multiple filler-gap dependencies and examine the role of case marking in the resolution of these dependencies. In a self-paced reading experiment, we discovered that the parallelism effects of both head nouns were instrumental in the resolution of longer dependencies of a higher head noun and its gap. The non-parallel role of a distractor (i.e., low head noun) was reanalyzed before the integration of higher head noun and its gap. This finding suggests that parallelism effects play a critical role in the resolution of complex dependency structures. Additionally, our experiments investigated the role of case marking as a morphosyntactic cue in predicting the syntactic encoding of upcoming noun phrases. Our results demonstrated the immediate processing difficulty of conditions with double nominative markers, suggesting that parsers are selectively sensitive to case marking. Overall, our study contributes to our understanding of the parsing process and the selective use of linguistic cues in the resolution of complex dependency structures.

A distribution-free change-point monitoring scheme in high-dimensional settings with application to industrial image surveillance

Existing monitoring tools for multivariate data are often asymptotically distribution-free, computationally intensive, or require a large stretch of stable data. Many of these methods are not applicable to ‘high-dimension, low sample size’ scenarios. With rapid technological advancement, high-dimensional data has become omnipresent in industrial applications. We propose a distribution-free change-point monitoring method applicable to high-dimensional data. Through an extensive simulation study, performance comparison has been done for different parameter values, under different multivariate distributions with complex dependence structures. The proposed method is robust and efficient in detecting change points under a wide range of shifts in the process distribution. A real-life application is illustrated with the help of a high-dimensional image surveillance dataset.

Flexible and consistent Flood–Duration–Frequency modeling: A Bayesian approach

Design flood values give estimates of flood magnitude within a given return period and are essential to making adaptive decisions around land use planning, infrastructure design, and disaster mitigation. Many hydrologic applications where flood retention is important, e.g. floodplain management and reservoir design, need design flood values for different durations. Flood–Duration–Frequency (QDF) models extend the standard statistical flood frequency analysis framework to multiple flood durations and are analogous to intensity–duration–frequency models for precipitation. Implementations of QDF models commonly assume simple scaling, where only the magnitude of the index flood is assumed to change with duration, despite empirical analyses showing a more complex dependence structure. We propose a multiscaling extension to existing QDF models where the magnitude of the index flood and the slope of the growth curve may scale independently with duration. In an application to 12 locations in Norway, we assess how three different QDF models capture relationships between floods of different duration. Incorporating duration dependency independently in both the index flood and the growth curve (extended QDF model) improves modeling of both short-duration events and events with long return periods. This model extension further expands the models’ ability to simultaneously model a wide range of durations. As measured by the integrated quadratic distance, the extended QDF model performs better than the original QDF model in 83% of the out of sample subdaily durations studied. Additionally, we find that the choice of durations used to fit QDF models is a highly influential aspect of the modeling process.

Solving Estimating Equations With Copulas

Thanks to their ability to capture complex dependence structures, copulas are frequently used to glue random variables into a joint model with arbitrary marginal distributions. More recently, they have been applied to solve statistical learning problems such as regression or classification. Framing such approaches as solutions of estimating equations, we generalize them in a unified framework. We can then obtain simultaneous, coherent inferences across multiple regression-like problems. We derive consistency, asymptotic normality, and validity of the bootstrap for corresponding estimators. The conditions allow for both continuous and discrete data as well as parametric, nonparametric, and semiparametric estimators of the copula and marginal distributions. The versatility of this methodology is illustrated by several theoretical examples, a simulation study, and an application to financial portfolio allocation. Supplementary materials for this article are available online.

R-statistic based predictor variables selection and vine structure determination approach for stochastic streamflow generation considering temporal and spatial dependence

Stochastic streamflow generation is crucial for water resources planning and management as well as water conservancy project design and operation. This study proposes an accurate, reliable and parsimonious approach for stochastic streamflow generation considering temporal and spatial dependence on the basis of regular vine copula model. The emphasis is on advancing an R-statistic based strategy of vine structure determination that divide the vine copula model construction into two independent parts and avoid continuous accumulation of uncertainty in the traditional Kendall's tau based method. Two study regions (the Upper Colorado River basin and Middle Yangtze River basin) with diverse hydrology regime and available data length are selected as case studies to showcase the performance of the proposed approach in practice. The results indicate better performance than two existing models in terms of streamflow estimation, and demonstrate that stochastic simulation series can preserve distribution and statistical characteristics of observed records. R-vine copula model constructed by the proposed approach is confirmed to possess low sensitivity to the number of predictor variables as well as good adaptability and robustness to streamflow series with diverse characteristics and abundances. The enhanced capability and performance stem from the accurate identification of predictor variables and characterization of complex and diverse dependence structures among different streamflow series, on the basis of a comprehensive and precise dependence measure, R-statistic.

The generalized ratios intrinsic dimension estimator

Modern datasets are characterized by numerous features related by complex dependency structures. To deal with these data, dimensionality reduction techniques are essential. Many of these techniques rely on the concept of intrinsic dimension (id), a measure of the complexity of the dataset. However, the estimation of this quantity is not trivial: often, the id depends rather dramatically on the scale of the distances among data points. At short distances, the id can be grossly overestimated due to the presence of noise, becoming smaller and approximately scale-independent only at large distances. An immediate approach to examining the scale dependence consists in decimating the dataset, which unavoidably induces non-negligible statistical errors at large scale. This article introduces a novel statistical method, Gride, that allows estimating the id as an explicit function of the scale without performing any decimation. Our approach is based on rigorous distributional results that enable the quantification of uncertainty of the estimates. Moreover, our method is simple and computationally efficient since it relies only on the distances among data points. Through simulation studies, we show that Gride is asymptotically unbiased, provides comparable estimates to other state-of-the-art methods, and is more robust to short-scale noise than other likelihood-based approaches.

Learning complex dependency structure of gene regulatory networks from high dimensional microarray data with Gaussian Bayesian networks

Reconstruction of Gene Regulatory Networks (GRNs) of gene expression data with Probabilistic Network Models (PNMs) is an open problem. Gene expression datasets consist of thousand of genes with relatively small sample sizes (i.e. are large-p-small-n). Moreover, dependencies of various orders coexist in the datasets. On the one hand transcription factor encoding genes act like hubs and regulate target genes, on the other hand target genes show local dependencies. In the field of Undirected Network Models (UNMs)—a subclass of PNMs—the Glasso algorithm has been proposed to deal with high dimensional microarray datasets forcing sparsity. To overcome the problem of the complex structure of interactions, modifications of the default Glasso algorithm have been developed that integrate the expected dependency structure in the UNMs beforehand. In this work we advocate the use of a simple score-based Hill Climbing algorithm (HC) that learns Gaussian Bayesian networks leaning on directed acyclic graphs. We compare HC with Glasso and variants in the UNM framework based on their capability to reconstruct GRNs from microarray data from the benchmarking synthetic dataset from the DREAM5 challenge and from real-world data from the Escherichia coli genome. We conclude that dependencies in complex data are learned best by the HC algorithm, presenting them most accurately and efficiently, simultaneously modelling strong local and weaker but significant global connections coexisting in the gene expression dataset. The HC algorithm adapts intrinsically to the complex dependency structure of the dataset, without forcing a specific structure in advance.

Robust High-Dimensional Regression with Coefficient Thresholding and Its Application to Imaging Data Analysis

It is important to develop statistical techniques to analyze high-dimensional data in the presence of both complex dependence and possible heavy tails and outliers in real-world applications such as imaging data analyses. We propose a new robust high-dimensional regression with coefficient thresholding, in which an efficient nonconvex estimation procedure is proposed through a thresholding function and the robust Huber loss. The proposed regularization method accounts for complex dependence structures in predictors and is robust against heavy tails and outliers in outcomes. Theoretically, we rigorously analyze the landscape of the population and empirical risk functions for the proposed method. The fine landscape enables us to establish both statistical consistency and computational convergence under the high-dimensional setting. We also present an extension to incorporate spatial information into the proposed method. Finite-sample properties of the proposed methods are examined by extensive simulation studies. An application concerns a scalar-on-image regression analysis for an association of psychiatric disorder measured by the general factor of psychopathology with features extracted from the task functional MRI data in the Adolescent Brain Cognitive Development (ABCD) study. Supplementary materials for this article are available online.