Empirical Marginal Distribution Research Articles

Variational Autoencoders and Generative Adversarial Networks for Multivariate Scenario Generation

When making decisions with lasting implications over a medium to long timeframe, it is essential to consider not only the most probable scenario, possibly obtained through a forecasting model, but also a range of potential outcomes. This approach allows for effective risk mitigation across a spectrum of scenarios, including less probable ones, and enhances the resilience of planning strategies. In this paper, we demonstrate the development of a generative model capable of learning the multivariate joint probability distribution of link speeds on a road network, using real sensor data. To further enhance the performance of our Generative Adversarial Network model, we employed a Variational AutoEncoder for pre-training the generator network. Experimental results, conducted on three distinct benchmark datasets, highlight the potential of the proposed model in generating new scenario samples of multivariate variables. The Wasserstein distance between the generated distribution and the real data, confirms the good performance of our model compared to state-of-the-art models, based on copulae. The proposed model has shown its ability to generate scenarios that preserve correlations among variables, while producing samples that faithfully represent the empirical marginal distributions.

Randomizing a clinical trial in neuro-degenerative disease

The paper studies randomization rules for a sequential two-treatment, two-site clinical trial in Parkinson’s disease. An important feature is that we have values of responses and five potential prognostic factors from a sample of 144 patients similar to those to be enrolled in the trial. Analysis of this sample provides a model for trial analysis. The comparison of allocation rules is made by simulation yielding measures of loss due to imbalance and of potential bias. A major novelty of the paper is the use of this sample, via a two-stage algorithm, to provide an empirical distribution of covariates for the simulation; sampling of a correlated multivariate normal distribution is followed by transformation to variables following the empirical marginal distributions. Six allocation rules are evaluated. The paper concludes with some comments on general aspects of the evaluation of such rules and provides a recommendation for two allocation rules, one for each site, depending on the target number of patients to be enrolled.

RafterNet: Probabilistic Predictions in Multi-Response Regression

A fully nonparametric approach for making probabilistic predictions in multi-response regression problems is introduced. Random forests are used as marginal models for each response variable and, as novel contribution of the present work, the dependence between the multiple response variables is modeled by a generative neural network. This combined modeling approach of random forests, corresponding empirical marginal residual distributions and a generative neural network is referred to as RafterNet. Multiple datasets serve as examples to demonstrate the flexibility of the approach and its impact for making probabilistic forecasts.

TACDFSL: Task Adaptive Cross Domain Few-Shot Learning

Cross Domain Few-Shot Learning (CDFSL) has attracted the attention of many scholars since it is closer to reality. The domain shift between the source domain and the target domain is a crucial problem for CDFSL. The essence of domain shift is the marginal distribution difference between two domains which is implicit and unknown. So the empirical marginal distribution measurement is proposed, that is, WDMDS (Wasserstein Distance for Measuring Domain Shift) and MMDMDS (Maximum Mean Discrepancy for Measuring Domain Shift). Besides this, pre-training a feature extractor and fine-tuning a classifier are used in order to have a good generalization in CDFSL. Since the feature obtained by the feature extractor is high-dimensional and left-biased, the adaptive feature distribution transformation is proposed, to make the feature distribution of each sample be approximately Gaussian distribution. This approximate symmetric distribution improves image classification accuracy by 3% on average. In addition, the applicability of different classifiers for CDFSL is investigated, and the classification model should be selected based on the empirical marginal distribution difference between the two domains. The Task Adaptive Cross Domain Few-Shot Learning (TACDFSL) is proposed based on the above ideas. TACDFSL improves image classification accuracy by 3–9%.

Bivariate bias correction of the regional climate model ensemble over the Adriatic region

AbstractThe main goal of this analysis is to provide an assessment of the fine grid regional climate model (RCM) ensemble over the broader Adriatic region, which is the domain with orographically complex terrain and a developed coastline. The results of this study are a beneficial complement to earlier work on using modified bivariate bias correction methods. This analysis was based on an ensemble of 12 combinations of three RCMs and four global climate models (GCMs) from 1971 to 2004, divided into calibration and validation parts. A comparison of one univariate and three versions of bivariate bias correction methods of summer and winter monthly temperature and precipitation was performed. To examine the effect of various methods, we used different marginal distributions and distributions of dependence (copula). Bivariate bias correction was conducted with parametric and empirical marginal distributions. In the quantile mapping (univariate) and the parametric bivariate method, gamma (precipitation) and normal (temperature) probability distributions were used. We documented the bias and the influence of bias correction methods on precipitation and temperature, including their climate change signals in historical data. The considered methods maintained the spatial distribution of trends from the native ensemble. If the relationship between variables was not distinct and the correlation coefficient was low, bivariate bias correction methods tended to weaken the correlation coefficient from the model. Two experiments on how the bias correction methods influence statistical measures of the considered variables and their relationships showed that the bivariate method with empirical distributions (eeG) performed slightly better than other methods. Although the eeG method performed better than the other methods, none of the considered methods was able to fully preserve the correlation coefficient between variables detected from observations. Considering two copulas (Gaussian and Student's t), we conclude that the effect of different copula types on method performance is small.

A data-driven method to learn a jump diffusion process from aggregate biological gene expression data

Dynamic models of gene expression are urgently required. In this paper, we describe the time evolution of gene expression by learning a jump diffusion process to model the biological process directly. Our algorithm needs aggregate gene expression data as input and outputs the parameters of the jump diffusion process. The learned jump diffusion process can predict population distributions of gene expression at any developmental stage, obtain long-time trajectories for individual cells, and offer a novel approach to computing RNA velocity. Moreover, it studies biological systems from a stochastic dynamic perspective. Gene expression data at a time point, which is a snapshot of a cellular process, is treated as an empirical marginal distribution of a stochastic process. The Wasserstein distance between the empirical distribution and predicted distribution by the jump diffusion process is minimized to learn the dynamics. For the learned jump diffusion process, its trajectories correspond to the development process of cells, the stochasticity determines the heterogeneity of cells, its instantaneous rate of state change can be taken as “RNA velocity”, and the changes in scales and orientations of clusters can be noticed too. We demonstrate that our method can recover the underlying nonlinear dynamics better compared to previous parametric models and the diffusion processes driven by Brownian motion for both synthetic and real world datasets. Our method is also robust to perturbations of data because the computation involves only population expectations.

Inducing a desired value of correlation between two point-scale variables: a two-step procedure using copulas

Focusing on point-scale random variables, i.e. variables whose support consists of the first m positive integers, we discuss how to build a joint distribution with pre-specified marginal distributions and Pearson’s correlation rho . After recalling how the desired value rho is not free to vary between -1 and +1, but generally ranges a narrower interval, whose bounds depend on the two marginal distributions, we devise a procedure that first identifies a class of joint distributions, based on a parametric family of copulas, having the desired margins, and then adjusts the copula parameter in order to match the desired correlation. The proposed methodology addresses a need which often arises when assessing the performance and robustness of some new statistical technique, i.e. trying to build a huge number of replicates of a given dataset, which satisfy—on average—some of its features (for example, the empirical marginal distributions and the pairwise linear correlations). The proposal shows several advantages, such as—among others—allowing for dependence structures other than the Gaussian and being able to accommodate the copula parameter up to an assigned level of precision for rho with a very small computational cost. Based on this procedure, we also suggest a two-step estimation technique for copula-based bivariate discrete distributions, which can be used as an alternative to full and two-step maximum likelihood estimation. Numerical illustration and empirical evidence are provided through some examples and a Monte Carlo simulation study, involving the CUB distribution and three different copulas; an application to real data is also discussed.

Geostatistical prediction through convex combination of Archimedean copulas

A common problem in geostatistics is to interpolate a variable at unsampled locations using available data. Kriging has been the conventional method of solving this problem by providing the weighted average of samples, which is determined by minimizing the estimation variance. Kriging variance is a function of the samples’ spatial configuration and the variable’s spatial dependence structure. The latter is described by covariance that reduces complex dependence structures of natural phenomena to a single measure, introducing substantial simplifications. Another issue about kriging is that its variance does not depend on the sample values. Therefore, applying new methods such as spatial copulas that better describe the spatial dependence structure of variables and take advantage of sample values and spatial dependence structure would be helpful. This study compares prediction through the convex combination of Archimedean copulas to kriging using seven variables of the Jura data set. The empirical marginal distribution of variables and fitted kernel density estimates based on the Gaussian, triangular, Epanechnikov and gamma functions were used to investigate the effects of margins on the results. The mixed copulas were capable of describing various types of dependencies with asymmetric upper and lower tails. However, the Gaussian copula failed to explain the spatial dependence structure of variables and had the worst results among the copula-based approaches. The application of empirical marginal distribution of variables has generally given better results than the fitted models. For variables with large ratios of nugget effect to sill, in general, the copula-based approaches showed an advantage over kriging due to better reproduction of the mean values and distributions of the variables, having lower mean squared errors and higher correlation coefficients between the predicted and observed values. On the other hand, for variables with small nugget effects, kriging has better performance regarding all criteria except for the mean value reproduction. This study suggests using a convex combination of Archimedean copulas to predict variables with significant nugget effects.

Checkerboard copula defined by sums of random variables

Abstract We consider the problem of finding checkerboard copulas for modeling multivariate distributions. A checkerboard copula is a distribution with a corresponding density defined almost everywhere by a step function on an m-uniform subdivision of the unit hyper-cube. We develop optimization procedures for finding copulas defined by multiply-stochastic matrices matching available information. Two types of information are used for building copulas: 1) Spearman Rho rank correlation coefficients; 2) Empirical distributions of sums of random variables combined with empirical marginal probability distributions. To construct checkerboard copulas we solved optimization problems. The first problem maximizes entropy with constraints on Spearman Rho coefficients. The second problem minimizes some error function to match available data. We conducted a case study illustrating the application of the developed methodology using property and casualty insurance data. The optimization problems were numerically solved with the AORDA Portfolio Safeguard (PSG) package, which has precoded entropy and error functions. Case study data, codes, and results are posted at the web.

Microbial Networks in SPRING - Semi-parametric Rank-Based Correlation and Partial Correlation Estimation for Quantitative Microbiome Data.

High-throughput microbial sequencing techniques, such as targeted amplicon-based and metagenomic profiling, provide low-cost genomic survey data of microbial communities in their natural environment, ranging from marine ecosystems to host-associated habitats. While standard microbiome profiling data can provide sparse relative abundances of operational taxonomic units or genes, recent advances in experimental protocols give a more quantitative picture of microbial communities by pairing sequencing-based techniques with orthogonal measurements of microbial cell counts from the same sample. These tandem measurements provide absolute microbial count data albeit with a large excess of zeros due to limited sequencing depth. In this contribution we consider the fundamental statistical problem of estimating correlations and partial correlations from such quantitative microbiome data. To this end, we propose a semi-parametric rank-based approach to correlation estimation that can naturally deal with the excess zeros in the data. Combining this estimator with sparse graphical modeling techniques leads to the Semi-Parametric Rank-based approach for INference in Graphical model (SPRING). SPRING enables inference of statistical microbial association networks from quantitative microbiome data which can serve as high-level statistical summary of the underlying microbial ecosystem and can provide testable hypotheses for functional species-species interactions. Due to the absence of verified microbial associations we also introduce a novel quantitative microbiome data generation mechanism which mimics empirical marginal distributions of measured count data while simultaneously allowing user-specified dependencies among the variables. SPRING shows superior network recovery performance on a wide range of realistic benchmark problems with varying network topologies and is robust to misspecifications of the total cell count estimate. To highlight SPRING's broad applicability we infer taxon-taxon associations from the American Gut Project data and genus-genus associations from a recent quantitative gut microbiome dataset. We believe that, as quantitative microbiome profiling data will become increasingly available, the semi-parametric estimators for correlation and partial correlation estimation introduced here provide an important tool for reliable statistical analysis of quantitative microbiome data.

Panic-aware portfolio optimization

This article provides a portfolio optimization approach that takes into account extreme events. By merging a (downside-only) panic copula with the empirical marginal distributions, panic-awareness is attained for the optimization process. This approach includes the likelihood of highly co-dependent asset movements in panic states of the market—as empirically observed during market crashes. Panic-awareness CVaR optimization translates into robust equity portfolios, empirically exemplified for the UK and German stock market.

Optimal Steering of a Linear Stochastic System to a Final Probability Distribution—Part III

The subject of this work has its roots in the so called Schroedginer Bridge Problem (SBP) which asks for the most likely distribution of Brownian particles in their passage between observed empirical marginal distributions at two distinct points in time. Renewed interest in this problem was sparked by a reformulation in the language of stochastic control. In earlier works, presented as Part I and Part II, we explored a generalization of the original SBP that amounts to optimal steering of linear stochastic dynamical systems between state-distributions, at two points in time, under full state feedback. In these works the cost was quadratic in the control input. The purpose of the present work is to detail the technical steps in extending the framework to the case where a quadratic cost in the state is also present. In the zero-noise limit, we obtain the solution of a (deterministic) mass transport problem with general quadratic cost.

Classification of pressure ulcer tissues with 3D convolutional neural network

A 3D convolution neural network (CNN) of deep learning architecture is supplied with essential visual features to accurately classify and segment granulation, necrotic eschar, and slough tissues in pressure ulcer color images. After finding a region of interest (ROI), the features are extracted from both the original and convolved with a pre-selected Gaussian kernel 3D HSI images, combined with first-order models of current and prior visual appearance. The models approximate empirical marginal probability distributions of voxel-wise signals with linear combinations of discrete Gaussians (LCDG). The framework was trained and tested on 193 color pressure ulcer images. The classification accuracy and robustness were evaluated using the Dice similarity coefficient (DSC), the percentage area distance (PAD), and the area under the ROC curve (AUC). The obtained preliminary DSC of 92%, PAD of 13%, and AUC of 95% are promising. Graphical Abstract The Classification of Pressure Ulcer Tissues Based on 3D Convolutional Neural Network.

Is Einsteinian no-signalling violated in Bell tests?

Abstract Relativistic invariance is a physical law verified in several domains of physics. The impossibility of faster than light influences is not questioned by quantum theory. In quantum electrodynamics, in quantum field theory and in the standard model relativistic invariance is incorporated by construction. Quantum mechanics predicts strong long range correlations between outcomes of spin projection measurements performed in distant laboratories. In spite of these strong correlations marginal probability distributions should not depend on what was measured in the other laboratory what is called shortly: non-signalling. In several experiments, performed to test various Bell-type inequalities, some unexplained dependence of empirical marginal probability distributions on distant settings was observed. In this paper we demonstrate how a particular identification and selection procedure of paired distant outcomes is the most probable cause for this apparent violation of no-signalling principle. Thus this unexpected setting dependence does not prove the existence of superluminal influences and Einsteinian no-signalling principle has to be tested differently in dedicated experiments. We propose a detailed protocol telling how such experiments should be designed in order to be conclusive. We also explain how magical quantum correlations may be explained in a locally causal way.

TEMPORAL SHAPING OF SIMULATED TIME SERIES WITH CYCLICAL SAMPLE PATHS

Temporal shaping of time series is the activity of deriving a time series model with a prescribed marginal distribution and some sample path characteristics. Starting with an empirical sample path, one often computes from it an empirical histogram (a step-function density) and empirical autocorrelation function. The corresponding cumulative distribution function is piecewise linear, and so is the inverse distribution function. The so-called inversion method uses the latter to generate the corresponding distribution from a uniform random variable on [0,1), histograms being a special case. This paper shows how to manipulate the inverse histogram and an underlying marginally uniform process, so as to “shape” the model sample paths in an attempt to match the qualitative nature of the empirical sample paths, while maintaining a guaranteed match of the empirical marginal distribution. It proposes a new approach to temporal shaping of time series and identifies a number of operations on a piecewise-linear inverse histogram function, which leave the marginal distribution invariant. For cyclical processes with a prescribed marginal distribution and a prescribed cycle profile, one can also use these transformations to generate sample paths which “conform” to the profile. This approach also improves the ability to approximate the empirical autocorrelation function.

Accurate Lungs Segmentation on CT Chest Images by Adaptive Appearance-Guided Shape Modeling.

To accurately segment pathological and healthy lungs for reliable computer-aided disease diagnostics, a stack of chest CT scans is modeled as a sample of a spatially inhomogeneous joint 3D Markov-Gibbs random field (MGRF) of voxel-wise lung and chest CT image signals (intensities). The proposed learnable MGRF integrates two visual appearance sub-models with an adaptive lung shape submodel. The first-order appearance submodel accounts for both the original CT image and its Gaussian scale space (GSS) filtered version to specify local and global signal properties, respectively. Each empirical marginal probability distribution of signals is closely approximated with a linear combination of discrete Gaussians (LCDG), containing two positive dominant and multiple sign-alternate subordinate DGs. The approximation is separated into two LCDGs to describe individually the lungs and their background, i.e., all other chest tissues. The second-order appearance submodel quantifies conditional pairwise intensity dependencies in the nearest voxel 26-neighborhood in both the original and GSS-filtered images. The shape submodel is built for a set of training data and is adapted during segmentation using both the lung and chest appearances. The accuracy of the proposed segmentation framework is quantitatively assessed using two public databases (ISBI VESSEL12 challenge and MICCAI LOLA11 challenge) and our own database with, respectively, 20, 55, and 30 CT images of various lung pathologies acquired with different scanners and protocols. Quantitative assessment of our framework in terms of Dice similarity coefficients, 95-percentile bidirectional Hausdorff distances, and percentage volume differences confirms the high accuracy of our model on both our database (98.4±1.0%, 2.2±1.0 mm, 0.42±0.10%) and the VESSEL12 database (99.0±0.5%, 2.1±1.6 mm, 0.39±0.20%), respectively. Similarly, the accuracy of our approach is further verified via a blind evaluation by the organizers of the LOLA11 competition, where an average overlap of 98.0% with the expert's segmentation is yielded on all 55 subjects with our framework being ranked first among all the state-of-the-art techniques compared.

Stochastic multiscale modeling of metal foams

We propose a procedure for the computation of natural frequencies for structures made of metal foam. Because the heterogeneity of the foam geometry has an influence on macroscopic properties, the irregular geometry of the foam has to be taken into account. This is done by adapting a Laguerre tessellation to statistical descriptors of the geometry obtained from CT image analysis. As the length scale of the representative volume element is nearly of the same order as the length scale of the structures under consideration, classical homogenization techniques for the computation of effective properties cannot be applied. Therefore, we introduce a stochastic homogenization method based on empirical marginal distribution functions and correlation functions of apparent properties. This information allows us to define random fields for elastic properties and the mass density on the macroscopic level. Statistical properties of the natural frequencies can then be inferred.

Aggregation of carbon dioxide sequestration storage assessment units

The U.S. Geological Survey is currently conducting a national assessment of carbon dioxide (CO2) storage resources, mandated by the Energy Independence and Security Act of 2007. Pre-emission capture and storage of CO2 in subsurface saline formations is one potential method to reduce greenhouse gas emissions and the negative impact of global climate change. Like many large-scale resource assessments, the area under investigation is split into smaller, more manageable storage assessment units (SAUs), which must be aggregated with correctly propagated uncertainty to the basin, regional, and national scales. The aggregation methodology requires two types of data: marginal probability distributions of storage resource for each SAU, and a correlation matrix obtained by expert elicitation describing interdependencies between pairs of SAUs. Dependencies arise because geologic analogs, assessment methods, and assessors often overlap. The correlation matrix is used to induce rank correlation, using a Cholesky decomposition, among the empirical marginal distributions representing individually assessed SAUs. This manuscript presents a probabilistic aggregation method tailored to the correlations and dependencies inherent to a CO2 storage assessment. Aggregation results must be presented at the basin, regional, and national scales. A single stage approach, in which one large correlation matrix is defined and subsets are used for different scales, is compared to a multiple stage approach, in which new correlation matrices are created to aggregate intermediate results. Although the single-stage approach requires determination of significantly more correlation coefficients, it captures geologic dependencies among similar units in different basins and it is less sensitive to fluctuations in low correlation coefficients than the multiple stage approach. Thus, subsets of one single-stage correlation matrix are used to aggregate to basin, regional, and national scales.

Mallows Distance in VARFIMA(0, d, 0) Processes

In this work we present an extensive simulation study on Mallows distance in the context of Gaussian and non Gaussian VARFIMA processes. Our main goal is to analyze the dependence among the components of VARFIMA processes through the Mallows distance point of view. A possible relationship between the Mallows distance and the fractional differencing parameter d , the type and level of dependence in the innovation process as well as its marginal behavior is investigated. We study the behavior of the Kendall's τ dependence coefficient under the same framework for comparison purposes. For the Mallows distance, we consider an estimator based on the empirical marginal distribution function. Based on our simulation results, we propose both a semiparametric estimator for the fractional differencing parameter and a testing procedure to assess the presence of strong long range dependence in the components of VARFIMA processes of any (finite) dimension.

Stochastic Multiscale Modeling of Metal Foams

A procedure for the computation of eigenfrequencies for structures made of metal foam is proposed. The heterogeneity of the foam geometry has an influence on these macroscopic properties and has to be taken into account. This is done by fitting a model of the microstructure based on Laguerre tessellations by means of statistical information obtained from CT image analysis. As the length scale of the representative volume element is nearly of the same order as the length scale of the structures under consideration, classical homogenization techniques for the computation of effective properties can not be applied. A stochastic homogenization method that necessitates the computation of empirical marginal distributions and correlation functions for apparent properties defined on the mesoscale is proposed. This information allows to define random fields for elastic properties and the density on the structure level. Statistical properties of the eigenfrequencies can then be inferred.