Empirical Marginal Distribution Research Articles

Probability Distribution of Rainfall-Runoff Using Entropy Theory

Using the entropy theory, the Shannon entropy-based joint probability distribution of rainfall amount and runoff depth (runoff volume per unit area) is derived. In order to evaluate the accuracy of the entropy-based joint distribution, the study applies the following approaches: (1) comparison of the entropy determined from the derived joint distribution with that of the empirical joint entropy calculated based on the kernel density with the optimized kernel bandwidth, (2) the root mean square error (RMSE) of the joint probability density derived from the joint distribution and the empirical joint density, and (3) testing the derived joint distribution using the chi-square goodness-of-fit test for multivariate distributions. In addition, the marginal distributions of rainfall and runoff derived from the joint distribution are compared with the empirical marginal and fitted gamma distributions and are evaluated using the Kolmogorov-Smirnov (K-S) goodness-of-fit test and RMSE. With the joint distribution appropriately determined, the conditional distribution of runoff depth is derived for a given rainfall depth. The derived distributions are evaluated using rainfall-runoff data from small agricultural experimental watersheds near Riesel (Waco), Texas, maintained by the USDA-ARS. Results of the study indicate that: (1) the Shannon entropy theory can be satisfactorily applied to model the bivariate rainfall and runoff distribution; (2) it can be applied to model the bivariate random variables when convenient bivariate distributions are not able to capture the dependence between them; and (3) because the Shannon entropy uses constraints to formulate the joint distribution, it may be applied to the bivariate random variables incorporating different types of marginal distributions.

A generalisation of independence in statistical models for categorical distribution

In this paper, generalised statistical independence in statistical models for categorical distributions is proposed from the viewpoint of generalised multiplication characterised by a monotonically increasing function and its inverse function, and it is implemented in naive Bayes models. This paper also proposes an idea of their estimation method which directly uses empirical marginal distributions to retain simplicity of calculation. This method is interpreted as an optimisation of a rough approximation of the Bregman divergence so that it is expected to have a kind of robust property. Effectiveness of proposed models is shown by numerical experiments on some benchmark datasets.

Stochastic estimation of biogeochemical parameters of a 3D ocean coupled physical–biogeochemical model: Twin experiments

In a 3D ocean coupled physical–biogeochemical model, implemented on the North Atlantic at 1/4° and including six biogeochemical variables, three parameters (phytoplankton maximal growth rate, phytoplankton mortality rate and zooplankton maximal grazing rate) are assumed to be stochastic and have regional variations. Ensemble simulations (200 members, lasting 30 days during the spring bloom) show that the phytoplankton concentration is sensitive to the parameterization, with strong spatial heterogeneity, combined to a nonlinear and non-Gaussian behavior. Within the Kalman filter theory, parameter estimation can be done, in the framework of optimal estimate with Gaussian assumptions and reduced rank approximation, when the state vector is augmented with the uncertain parameters. Twin data assimilation experiments, using surface phytoplankton as observations, were performed either in the linear framework or introducing a nonlinear local transformation (anamorphosis). The anamorphosis is performed using a piecewise linear change of variables (applied to all biogeochemical quantities) remapping the percentiles of the empirical marginal distribution provided by the ensemble on the percentiles of the Gaussian distribution. Nonlinear parameter estimation performed better than linear estimation: on the 39 estimated parameters, there is a reduction in the variance obtained with the nonlinear analysis, compared to the variance obtained with the linear analysis, except for 2 parameters. The reduction is better than 60% in 80% of these cases. The anamorphosis is also useful to define an objective error norm for the biogeochemical variables.

Statistical Modeling of 3-D Natural Scenes With Application to Bayesian Stereopsis

We studied the empirical distributions of luminance, range and disparity wavelet coefficients using a coregistered database of luminance and range images. The marginal distributions of range and disparity are observed to have high peaks and heavy tails, similar to the well-known properties of luminance wavelet coefficients. However, we found that the kurtosis of range and disparity coefficients is significantly larger than that of luminance coefficients. We used generalized Gaussian models to fit the empirical marginal distributions. We found that the marginal distribution of luminance coefficients have a shape parameter p between 0.6 and 0.8, while range and disparity coefficients have much smaller parameters p < 0.32, corresponding to a much higher peak. We also examined the conditional distributions of luminance, range and disparity coefficients. The magnitudes of luminance and range (disparity) coefficients show a clear positive correlation, which means, at a location with larger luminance variation, there is a higher probability of a larger range (disparity) variation. We also used generalized Gaussians to model the conditional distributions of luminance and range (disparity) coefficients. The values of the two shape parameters (p,s) reflect the observed luminance-range (disparity) dependency. As an example of the usefulness of luminance statistics conditioned on range statistics, we modified a well-known Bayesian stereo ranging algorithm using our natural scene statistics models, which improved its performance.

How has Educational Expansion Changed the Necessary and Sufficient Conditions for Achieving Professional, Managerial and Technical Class Positions in Britain? A Configurational Analysis

This paper, whose purpose is both substantive and methodological, focuses on changes over a nine year period, drawing on data from two British birth cohorts (individuals born in 1958 and 1970), and, substantively, employs set theoretic methods to explore the extent to which an upward shift in qualifications achieved led to any reduction in the roles class and gender played in the achievement of professional, managerial and technical (PMT) social class destinations in early adulthood. Our methodological purpose is to illustrate how a counterfactual modelling approach can be used together with Ragin's set theoretic methods to provide an alternative way of analysing relationships in this area. We draw on earlier work exploring the extent to which educational achievement was ‘meritocratic’ with respect to ability for these cohorts (Cooper 2005, 2006). Our configurational account of the causal pathways to various class destinations is set against the background of a simple model of ‘meritocracy’ (allocation to available class positions by qualifications alone taking account of the empirical marginal distributions). This model allows us to specify, counterfactually, what qualifications would have represented necessary and sufficient conditions in our modelled meritocracy for reaching the PMT class. By comparison of these conditions with the empirically derived necessary and sufficient conditions for achieving these outcomes (using Ragin et al's fs/QCA software) we show that while allocation processes were far from meritocratic in both cohorts, there were some changes in the way both class and gender combined with qualifications as conditions for destinations. We also show that Ragin's configurational methods, focussing on holistically-conceived cases and conjunctural causation rather than on the net effects of independent variables, provide a useful analytic technique for capturing relationships in this field.

Precise segmentation of multimodal images

We propose new techniques for unsupervised segmentation of multimodal grayscale images such that each region-of-interest relates to a single dominant mode of the empirical marginal probability distribution of grey levels. We follow the most conventional approaches in that initial images and desired maps of regions are described by a joint Markov-Gibbs random field (MGRF) model of independent image signals and interdependent region labels. However, our focus is on more accurate model identification. To better specify region borders, each empirical distribution of image signals is precisely approximated by a linear combination of Gaussians (LCG) with positive and negative components. We modify an expectation-maximization (EM) algorithm to deal with the LCGs and also propose a novel EM-based sequential technique to get a close initial LCG approximation with which the modified EM algorithm should start. The proposed technique identifies individual LCG models in a mixed empirical distribution, including the number of positive and negative Gaussians. Initial segmentation based on the LCG models is then iteratively refined by using the MGRF with analytically estimated potentials. The convergence of the overall segmentation algorithm at each stage is discussed. Experiments show that the developed techniques segment different types of complex multimodal medical images more accurately than other known algorithms.

Simulation of Highly Skewed Non-Gaussian Stochastic Processes

In this paper, a simulation methodology is proposed to generate sample functions of a stationary, non-Gaussian stochastic process with prescribed spectral density function and prescribed marginal probability distribution. The proposed methodology is a modified version of the Yamazaki and Shinozuka iterative algorithm that has certain difficulties matching the prescribed marginal probability distribution. Although these difficulties are usually sufficiently small when simulating non-Gaussian stochastic processes with slightly skewed marginal probability distributions, they become more pronounced for highly skewed probability distributions (especially at the tails of such distributions). Two major modifications are introduced in the original Yamazaki and Shinozuka iterative algorithm to ensure a practically perfect match of the prescribed marginal probability distribution regardless of the skewness of the distribution considered. First, since the underlying “Gaussian” stochastic process from which the desired non-Gaussian process is obtained as a translation process becomes non-Gaussian after the first iteration, the empirical (non-Gaussian) marginal probability distribution of the underlying stochastic process is calculated at each iteration. This empirical non-Gaussian distribution is then used instead of the Gaussian to perform the nonlinear mapping of the underlying stochastic process to the desired non-Gaussian process. This modification ensures that at the end of the iterative scheme every generated non-Gaussian sample function will have the exact prescribed non-Gaussian marginal probability distribution. Second, before the start of the iterative scheme, a procedure named “spectral preconditioning” is carried out to check the compatibility between the prescribed spectral density function and prescribed marginal probability distribution. If these two quantities are found to be incompatible, then the spectral density function can be slightly modified to make it compatible with the prescribed marginal probability distribution. Finally, numerical examples (including a stochastic process with a highly skewed marginal probability distribution) are provided to demonstrate the capabilities of the proposed algorithm.

Arm processes and modeling methodology

ARM (Auto-Regressive Modular) processes constitute a broad class of nonlinear autoregressive schemes with modulo-1 reduction and additional transformations. Unlike their TES (Transform-Expand-Sample) precursors, which only admit iid innovation sequences, ARM processes admit dependent innovation sequences as well, so long as they are independent of the initial ARM variate. As such, the class of ARM processes constitutes a considerable generalization of the TES class, endowed with enhanced modeling flexibility. For example, a Markovian innovation sequence can model burstiness in traffic processes far better by making use of the Markovian state to capture the structure of bursts. This paper introduces ARM processes and derives their fundamental properties in terms of marginal distributions and autocorrelation functions. It defines several useful subclasses that illustrate the modeling flexibility of ARM classes. Finally, it outlines a modeling methodology of empirical time series that can simultaneously fit the empirical marginal distribution (histogram) and empirical autocorrelation function.

The empirical TES methodology: modeling empirical time series

TES (Transform‐Expand‐Sample) is a versatile class of stochastic sequences defined via an autoregressive scheme with modulo‐1 reduction and additional transformations. The scope of TES encompasses a wide variety of sample path behaviors, which in turn give rise to autocorrelation functions with diverse functional forms ‐ monotone, oscillatory, alternating, and others. TES sequences are readily generated on a computer, and their autocorrelation functions can be numerically computed from accurate analytical formulas at a modest computational cost.This paper presents the empirical TES modeling methodology which uses TES process theory to model empirical records. The novel feature of the TES methodology is that it expressly aims to simultaneously capture the empirical marginal distribution (histogram) and autocorrelation function. We draw attention to the non‐parametric nature of TES modeling in that it always guarantees an exact match to the empirical marginal distribution. However, fitting the corresponding autocorrelation function calls for a heuristic search for a TES model over a large parametric space. Consequently, practical TES modeling of empirical records must currently rely on software assistance. A visual interactive software environment, called TEStool, has been designed and implemented to support TES modeling. The paper describes the empirical TES modeling methodology as implemented in TEStool and provides numerically‐computable formulas for TES autocorrelations. Two examples illustrate the efficacy of the TES modeling approach. These examples serve to highlight the ability of TES models to capture first‐order and second‐order properties of empirical sample paths and to mimic their qualitative appearance.

A Survey of TES Modeling Applications

TES (Transform-Expand-Sample) is a versatile methodology for modeling general stationary time series, and particularly those that are autocorrelated. From the viewpoint of Monte Carlo simulation, TES represents a new and flexible input analysis approach. The salient feature of TES is its potential ability to simultaneously capture first-order and second-order properties of empirical time series (field measurements): Given an empirical sample, TES is designed to fit an arbitrary empirical marginal distribution (histogram) and to simultaneously approximate the leading empirical autocorrelations. Practical TES modeling is computationally intensive and can be effectively carried out only with computer support. A software modeling environment, called TEStool, has been designed and implemented to support the TES modeling methodology, through an interactive heuristic or algorithmic search approach employing extensive visualization. The purpose of this paper is to introduce TES modeling, and to offer some illustrative examples from a range of applications, including source modeling of compressed video and fault arrivals, financial modeling and texture generation. These examples demonstrate the efficacy and versatility of the TES modeling methodology, and underscore the high fidelity attainable with TES models.

Regenerative simulation of TES processes

Regenerative simulation has become a familiar and established tool for simulation-based estimation. However, many applications (e.g., traffic in high-speed communications networks) call for autocorrelated stochastic models to which traditional regenerative theory is not directly applicable. Consequently, extensions of regenerative simulation to dependent time series is increasingly gaining in theoretical and practical interest, with Markov chains constituting an important case. Fortunately, a regenerative structure can be identified in Harris-recurrent Markov chains with minor modification, and this structure can be exploited for standard regenerative estimation. In this paper we focus on a versatile class of Harris-recurrent Markov chains, called TES (Transform-Expand-Sample). TES processes can generate a variety of sample paths with arbitrary marginal distributions, and autocorrelation functions with a variety of functional forms (monotone, oscillating and alternating). A practical advantage of TES processes is that they can simultaneously capture the first and second order statistics of empirical sample paths (raw field measurements). Specifically, the TES modeling methodology can simultaneously match the empirical marginal distribution (histogram), as well as approximate the empirical autocorrelation function. We explicitly identify regenerative structures in TES processes and proceed to address efficiency and accuracy issues of prospective simulations. To show the efficacy of our approach, we report on a TES/M/1 case study. In this study, we used the likelihood ratio method to calculate the mean waiting time performance as a function of the regenerative structure and the intrinsic TES parameter controlling burstiness (degree of autocorrelation) in the arrival process. The score function method was used to estimate the corresponding sensitivity (gradient) with respect to the service rate. Finally, we demonstrated the importance of the particular regenerative structure selected in regard to the estimation efficiency and accuracy induced by the regeneration cycle length.