Arbitrary Marginal Distributions Research Articles

PyCoSMoS: An advanced toolbox for simulating real-world hydroclimatic data

Simulation models are a fundamental tool for investigating hydrological processes and for water resource management. In this study, we introduce PyCoSMoS, a Python toolbox that enables researchers to simulate observed univariate time series mimicking hydroclimatic processes. This toolbox preserves arbitrary marginal distribution and autocorrelation functions, while significantly reducing computational burden. PyCoSMoS is built upon the mixed-Uniform CoSMoS method recently proposed by Papalexiou et al. (2023). The toolbox is designed to minimize the user’s input, requiring only observed time series, marginal distribution, correlation function, and the number of lags. The output provides both visual and quantitative comparisons between the observed and simulated time series. We evaluate the performance of the package using various synthetic case studies and the results demonstrate satisfactory accuracy. Furthermore, we apply the toolbox to three real case studies: precipitation, temperature, and relative humidity, for which the toolbox can successfully simulate the observed time series in each case.

A flexible Clayton-like spatial copula with application to bounded support data

The Gaussian copula is a powerful tool that has been widely used to model spatial and/or temporal correlated data with arbitrary marginal distributions. However, this kind of model can potentially be too restrictive since it expresses a reflection symmetric dependence. In this paper, we propose a new spatial copula model that makes it possible to obtain random fields with arbitrary marginal distributions with a type of dependence that can be reflection symmetric or not.Particularly, we propose a new random field with uniform marginal distributions that can be viewed as a spatial generalization of the classical Clayton copula model. It is obtained through a power transformation of a specific instance of a beta random field which in turn is obtained using a transformation of two independent Gamma random fields.For the proposed random field, we study the second-order properties and we provide analytic expressions for the bivariate distribution and its correlation. Finally, in the reflection symmetric case, we study the associated geometrical properties. As an application of the proposed model we focus on spatial modeling of data with bounded support. Specifically, we focus on spatial regression models with marginal distribution of the beta type. In a simulation study, we investigate the use of the weighted pairwise composite likelihood method for the estimation of this model. Finally, the effectiveness of our methodology is illustrated by analyzing point-referenced vegetation index data using the Gaussian copula as benchmark. Our developments have been implemented in an open-source package for the R statistical environment.

A time variant uncertainty propagation method for high-dimensional dynamic structural system via K-L expansion and Bayesian deep neural network.

In this paper, a time variant uncertainty propagation (TUP) method for dynamic structural system with high-dimensional input variables is proposed. Firstly, an arbitrary stochastic process simulation (ASPS) method based on Karhunen-Loève (K-L) expansion and numerical integration is developed, expressing the stochastic process as the combination of its marginal distributions and eigen functions at several discrete time points. Secondly, the iterative sorting method is implemented to the statistic samples of marginal distributions for matching the constraints of covariance function. Since marginal distributions are directly used to express the stochastic process, the proposed ASPS is suitable for stationary or non-stationary stochastic processes with arbitrary marginal distributions. Thirdly, the high-dimensional TUP problem is converted into several high-dimensional static uncertainty propagation (UP) problems after implementing ASPS. Then, the Bayesian deep neural network based UP method is used to compute the marginal distributions as well as the eigen functions of dynamic system response, the high-dimensional TUP problem can thus be solved. Finally, several numerical examples are used to validate the effectiveness of the proposed method. This article is part of the theme issue 'Physics-informed machine learning and its structural integrity applications (Part 1)'.

Accounting for PD-LGD dependency: A tractable extension to the Basel ASRF framework

We extend the asymptotic single risk factor (ASRF) model used in the Basel regulations to accommodate the dependence between the probabilities of default (PD) and the losses given default (LGD) with arbitrary marginal distributions. The PD-LGD link is introduced via the single systematic risk factor and its strength is controlled via a dedicated parameter, in line with the treatment of the default dependence in the current Basel framework. We derive the explicit form of the mapping function translating unconditional LGDs into conditional ones, compute the portfolio-invariant semi-analytical formula of value-at-risk and propose a calibration method. An empirical study featuring defaulted corporate bonds confirms the validity of the calibration procedure and delivers realistic risk metrics. The proposed approach is easy to implement and is useful to design future guidelines for capital requirements, to perform sensitivity analysis or to compute implied downturn LGDs. Compared to the guidelines of the European Banking Authority, our approach delivers more conservative figures on the considered data.

Solving Estimating Equations With Copulas

Thanks to their ability to capture complex dependence structures, copulas are frequently used to glue random variables into a joint model with arbitrary marginal distributions. More recently, they have been applied to solve statistical learning problems such as regression or classification. Framing such approaches as solutions of estimating equations, we generalize them in a unified framework. We can then obtain simultaneous, coherent inferences across multiple regression-like problems. We derive consistency, asymptotic normality, and validity of the bootstrap for corresponding estimators. The conditions allow for both continuous and discrete data as well as parametric, nonparametric, and semiparametric estimators of the copula and marginal distributions. The versatility of this methodology is illustrated by several theoretical examples, a simulation study, and an application to financial portfolio allocation. Supplementary materials for this article are available online.

Large‐Domain Multisite Precipitation Generation: Operational Blueprint and Demonstration for 1,000 Sites

AbstractStochastic simulations of spatiotemporal patterns of hydroclimatic processes, such as precipitation, are needed to build alternative but equally plausible inputs for water‐related design and management, and to estimate uncertainty and assess risks. However, while existing stochastic simulation methods are mature enough to deal with relatively small domains and coarse spatiotemporal scales, additional work is required to develop simulation tools for large‐domain analyses, which are more and more common in an increasingly interconnected world. This study proposes a methodological advancement in the CoSMoS framework, which is a flexible simulation framework preserving arbitrary marginal distributions and correlations, to dramatically decrease the computational burden and make the algorithm fast enough to perform large‐domain simulations in short time. The proposed approach focuses on correlated processes with mixed (zero‐inflated) Uniform marginal distributions. These correlated processes act as intermediates between the target process to simulate (precipitation) and parent Gaussian processes that are the core of the simulation algorithm. Working in the mixed‐Uniform space enables a substantial simplification of the so‐called correlation transformation functions, which represent a computational bottle neck in the original CoSMoS formulation. As a proof of concept, we simulate 40 years of daily precipitation records from 1,000 gauging stations in the Mississippi River basin. Moreover, we extend CoSMoS incorporating parent non‐Gaussian processes with different degrees of tail dependence and suggest potential improvements including the separate simulation of occurrence and intensity processes, and the use of advection, anisotropy, and nonstationary spatiotemporal correlation functions.

Development of a model for stochastic analysis of vehicle–track interaction considering spatial variability of track parameters

In this work, a model for achieving stochastic analysis of vehicle–track interaction systems is proposed based on random field realization of track parameters and the finite element coupling of rigid-elastic bodies. Aiming at the clarification of the spatial variability and correlation of track parameters on the influence of vehicle–track dynamic performance, a general scheme based on Karhunen–Loève Expansion and cyclic calculation method is developed to realize a multi-dimensional random field that holds an arbitrary marginal distribution function and correlation structure, and to consider the inhomogeneity of system parameters along the longitudinal direction efficiently. Furthermore, the validation of the random field expansion against the vehicle–track system is implemented, and the dynamic effects of random track parameters are investigated through two special issues. Numerical results show that the spatial correlation of fastener stiffness affects the extreme response, especially the low-frequency component of rail displacement. The spatial dependence of the filling layer has little influence on the dynamic behavior of the vehicle subsystem, but it obviously changes the extreme stress and strain of the filling layer. Ignoring the spatial correlation of random variables might underestimate the performance of the track system.

A Generative Model for Correlated Graph Signals

A graph signal is a random vector with a partially known statistical description. The observations are usually sufficient to determine marginal distributions of graph node variables and their pairwise correlations representing the graph edges. However, the curse of dimensionality often prevents estimating a full joint distribution of all variables from the available observations. This paper introduces a computationally effective generative model to sample from arbitrary but known marginal distributions with defined pairwise correlations. Numerical experiments show that the proposed generative model is generally accurate for correlation coefficients with magnitudes up to about 0.3, whilst larger correlations can be obtained at the cost of distribution approximation accuracy. The generative models of graph signals can also be used to sample multivariate distributions for which closed-form mathematical expressions are not known or are too complex.

Spatial Hurst–Kolmogorov Clustering

The stochastic analysis in the scale domain (instead of the traditional lag or frequency domains) is introduced as a robust means to identify, model and simulate the Hurst–Kolmogorov (HK) dynamics, ranging from small (fractal) to large scales exhibiting the clustering behavior (else known as the Hurst phenomenon or long-range dependence). The HK clustering is an attribute of a multidimensional (1D, 2D, etc.) spatio-temporal stationary stochastic process with an arbitrary marginal distribution function, and a fractal behavior on small spatio-temporal scales of the dependence structure and a power-type on large scales, yielding a high probability of low- or high-magnitude events to group together in space and time. This behavior is preferably analyzed through the second-order statistics, and in the scale domain, by the stochastic metric of the climacogram, i.e., the variance of the averaged spatio-temporal process vs. spatio-temporal scale.

Development of a data generator for multivariate numerical data with arbitrary correlations and distributions

Artificial or simulated data are particularly relevant in tests and benchmarks for machine learning methods, in teaching for exercises and for setting up analysis workflows. They are relevant when real data may not be used for reasons of data protection, or when special distributions or effects should be present in the data to test certain machine learning methods. In this paper a generator for multivariate numerical data with arbitrary marginal distributions and – as far as possible – arbitrary correlations is presented. The data generator is implemented in the open source statistics software R. It can also be used for categorical variables, if data are generated separately for the corresponding characteristics of a categorical variable. Additionally, outliers can be integrated. The use of the data generator is demonstrated with a concrete example.

A counterexample to the existence of a general central limit theorem for pairwise independent identically distributed random variables

The classical Central Limit Theorem (CLT) is one of the most fundamental results in statistics. It states that the standardized sample mean of a sequence of n mutually independent and identically distributed random variables with finite second moment converges in distribution to a standard Gaussian as n goes to infinity. In particular, pairwise independence of the sequence is generally not sufficient for the theorem to hold. We construct explicitly such a sequence of pairwise independent random variables having a common but arbitrary marginal distribution F (satisfying very mild conditions) and for which no CLT holds. We obtain, in closed form, the asymptotic distribution of the sample mean of our sequence, and find it is asymmetrical for any F. This is illustrated through several theoretical examples for various choices of F. Associated R codes are provided in a supplementary appendix online.

Modelling Volatile Time Series with V-Transforms and Copulas

An approach to the modelling of volatile time series using a class of uniformity-preserving transforms for uniform random variables is proposed. V-transforms describe the relationship between quantiles of the stationary distribution of the time series and quantiles of the distribution of a predictable volatility proxy variable. They can be represented as copulas and permit the formulation and estimation of models that combine arbitrary marginal distributions with copula processes for the dynamics of the volatility proxy. The idea is illustrated using a Gaussian ARMA copula process and the resulting model is shown to replicate many of the stylized facts of financial return series and to facilitate the calculation of marginal and conditional characteristics of the model including quantile measures of risk. Estimation is carried out by adapting the exact maximum likelihood approach to the estimation of ARMA processes, and the model is shown to be competitive with standard GARCH in an empirical application to Bitcoin return data.

Counterexamples to the classical central limit theorem for triplewise independent random variables having a common arbitrary margin

AbstractWe present a general methodology to construct triplewise independent sequences of random variables having a common but arbitrary marginal distributionF(satisfying very mild conditions). For two specific sequences, we obtain in closed form the asymptotic distribution of the sample mean. It isnon-Gaussian(and depends on the specific choice ofF). This allows us to illustrate the extent of the ‘failure’ of the classical central limit theorem (CLT) under triplewise independence. Our methodology is simple and can also be used to create, for any integerK, newK-tuplewise independent sequences that are not mutually independent. ForK[four.tf], it appears that the sequences created using our methodologydoverify a CLT, and we explain heuristically why this is the case.

Parameter Estimation for Multi-state Coherent Series and Parallel Systems with Positively Quadrant Dependent Models

A multi-state coherent system consists of multiple components each of which passes through a sequence of states and it is of interest to estimate the distribution of time spent by the components in different states. Although not practical, for mathematical convenience, it is usually assumed that the times spent by the components in various states are independent of each other. This paper considers three-state series and parallel systems and is based on the assumption that times spent by the components in various states are positively quadrant dependent (PQD) and the corresponding dependence is modeled using a Farlie Gumbel Morgenstern (FGM) distribution. To begin with it is shown that even when marginal distributions are assumed exponential, the resulting likelihood function leads to a complicated expression making maximum likelihood (ML) based inference computationally challenging. A generalized method of moment (GMM) estimation is shown to be relatively simpler not only computationally but also the method works for arbitrary marginal distributions. The estimates obtained by GMM are shown to be uniformly consistent under some mild regularity conditions. Finite sample performances of the ML and GMM are illustrated using FGM distribution with various parametric marginal distributions. In case of exponential marginals, it is shown that GMM compares favorably to ML although the former method does not require parametric assumption for the marginals. The proposed methods are also illustrated using a real case study data of a rare type of head and neck cancer.

A Counterexample to the Central Limit Theorem for Pairwise Independent Random Variables Having a Common Absolutely Continuous Arbitrary Margin

The Central Limit Theorem (CLT) is one of the most fundamental results in Statistics. It states that the standardized sample mean of a sequence of n mutually independent and identically distributed random variables with finite first and second moments converges in distribution to a standard Gaussian as n goes to infinity. In particular, pairwise independence of the sequence is generally not sufficient for the theorem to hold. In this paper, we review the literature on sequences of random variables that fail to satisfy the conclusion of the CLT. Additionally, we construct explicitly a sequence of pairwise independent random variables having a common but arbitrary marginal distribution $\mathcal{L}$, and for which the CLT is not verified. We study the extent of this 'failure' of the CLT by obtaining, in closed form, the asymptotic distribution of the sample mean of our sequence. It is asymmetric with a tail that is always heavier than that of a Gaussian. It is remarkable that this asymptotic distribution is parametric, its sole parameter being related to the median of $\mathcal{L}$.

Simple and efficient adaptive two-sample tests for high-dimensional data

In this paper, we first consider testing the equality of two mean vectors in the so-called “high-dimension, low sample size” setting. Different from the existing methods in the literature, we propose a new adaptive test for this two-sample problem based on the order statistics of component-wise two-sample t-statistics. The proposed test is easy to implement and is carried out using permutations, therefore it does not rely on assumptions about normality or the structure of the covariance matrix. Furthermore, our simulation studies show that the proposed test is efficient for detecting different types of differences between two mean vectors, and compares favorably with the existing adaptive tests across a variety of settings. Using the same idea of incorporating the order statistics into the test, we further propose an adaptive two-sample test for testing location differences when marginal distributions have heavy-tails and another adaptive two-sample test for testing arbitrary marginal distribution differences, given high-dimensional, low sample size data. We also demonstrate the usefulness of these tests in high-dimensional, low sample size situations using simulated data as well as real data.

On the Exact Distribution of Correlated Extremes in Hydrology

AbstractThe analysis of hydrological hazards usually relies on asymptotic results of extreme value theory, which commonly deals with block maxima or peaks over threshold (POT) data series. However, data quality and quantity of block maxima and POT hydrological records do not usually fulfill the basic requirements of extreme value theory, thus making its application questionable and results prone to high uncertainty and low reliability. An alternative approach to better exploit the available information of continuous time series and nonextreme records is to build the exact distribution of maxima (i.e., nonasymptotic extreme value distributions) from a sequence of low‐threshold POT. Practical closed‐form results for this approach do exist only for independent high‐threshold POT series with Poisson occurrences. This study introduces new closed‐form equations of the exact distribution of maxima taken from low‐threshold POT with magnitudes characterized by an arbitrary marginal distribution and first‐order Markovian dependence, and negative binomial occurrences. The proposed model encompasses and generalizes the independent‐Poisson model and allows for analyses relying on significantly larger samples of low‐threshold POT values exhibiting dependence, temporal clustering, and overdispersion. To check the analytical results, we also introduce a new generator (called Gen2Mp) of proper first‐order Markov chains with arbitrary marginal distributions. An illustrative application to long‐term rainfall and streamflow data series shows that our model for the distribution of extreme maxima under dependence takes a step forward in developing more reliable data‐rich‐based analyses of extreme values.

Reliability assessment for multi-state systems with state transition dependency

As multi-state system reliability models are capable of characterizing the multi-state nature of engineered systems in deteriorating process, they have received considerable concerns in the past few decades. Most reported works on multi-state system reliability modeling are, however, based on the premise that transitions among states of a system/component are stochastically independent. Sometimes, a system may experience the same environmental/working conditions when deteriorating from a better state to worse ones, and thus, state transitions of a system/component could be stochastically dependent. In this paper, a new reliability model for multi-state systems/components with state transition dependency is put forth. The dependency among state transitions is implicitly characterized by copula functions which offer a great flexibility of linking arbitrary marginal distributions together to construct a multivariate distribution. The reliability function of a population of multi-state systems/components and the conditional reliability function of each individual multi-state system/component given a set of observations are derived. The likelihood functions for model parameter estimation are formulated for two cases, i.e., continuous and discontinuous inspection strategies, and model selection criteria are customized to identify the most preferable model among candidates. Two illustrative examples, together with comparative studies, are presented to demonstrate the effectiveness of the proposed method.

Dependence modelling in ultra high dimensions with vine copulas and the Graphical Lasso

To model high dimensional data, Gaussian methods are widely used since they remain tractable and yield parsimonious models by imposing strong assumptions on the data. Vine copulas are more flexible by combining arbitrary marginal distributions and (conditional) bivariate copulas. Yet, this adaptability is accompanied by sharply increasing computational effort as the dimension increases. The proposed approach overcomes this burden and makes the first step into ultra high dimensional non-Gaussian dependence modelling by using a divide-and-conquer approach. First, Gaussian methods are applied to split datasets into feasibly small subsets and second, parsimonious and flexible vine copulas are applied thereon. Finally, these sub-models are reconciled into one joint model. Numerical results demonstrating the feasibility of the novel approach in moderate dimensions are provided. The ability of the approach to estimate ultra high dimensional non-Gaussian dependence models in thousands of dimensions is presented.

Simulation of Stochastic Processes Exhibiting Any‐Range Dependence and Arbitrary Marginal Distributions

AbstractHydrometeorological processes are typically characterized by temporal dependence, short‐ or long‐range (e.g., Hurst behavior), as well as by non‐Gaussian distributions (especially at fine time scales). The generation of long synthetic time series that resembles the marginal and joint properties of the observed ones is a prerequisite in many uncertainty‐related hydrological studies, since they can be used as inputs and hence allow the propagation of natural variability and uncertainty to the typically deterministic water‐system models. For this reason, it has been for years one of the main research topics in the field of stochastic hydrology. This work presents a novel model for synthetic time series generation, termed Symmetric Moving Average (neaRly) To Anything, that holds out the promise of simulating stationary univariate and multivariate processes with any‐range dependence and arbitrary marginal distributions, provided that the former is feasible and the latter have finite variance. This is accomplished by utilizing a mapping procedure in combination with the relationship that exists between the correlation coefficients of an auxiliary Gaussian process and a non‐Gaussian one, formalized through the Nataf's joint distribution model. The generality of Symmetric Moving Average (neaRly) To Anything is stressed through two hypothetical simulation studies (univariate and multivariate), characterized by different dependencies and distributions. Furthermore, we demonstrate the practical aspects of the proposed model through two real‐world cases, one that concerns the generation of annual non‐Gaussian streamflow time series at four stations and another that involves the synthesis of intermittent, non‐Gaussian, daily rainfall series at a single location.