Joint Distribution Of Variables Research Articles

Disentangled latent factors for muti-cause treatment effect estimation

Abstract Existing methods estimate treatment effects from observational data and assume that covariates are all confounders. However, observed covariates may not directly represent confounding variables that influence both treatment and outcome. They always include variables that only affect the treatment or the outcome. In addition, for multi-dimensional binary treatments, disentangled methods are mainly designed for binary variables and ignore the impact of multi-cause treatment variables on the inference of latent factors. To address these two issues, based on variable decomposition and proxy inference, we propose the Disentangled Latent Factors for Multi-cause Treatment Estimation (DEMTE) algorithm. It utilizes an identifiable autoencoder to infer and disentangle latent factors based on the joint distribution of variables in observational data. DEMTE evaluates the treatment effect on the disentangled factors. Synthetic experiments and semi-synthetic experiments demonstrate the effectiveness of the inference and disentanglement techniques and our method achieves more accurate treatment effect estimation.

Estimating Multivariate Discrete Distributions Using Bernstein Copulas.

Measuring the dependence between random variables is one of the most fundamental problems in statistics, and therefore, determining the joint distribution of the relevant variables is crucial. Copulas have recently become an important tool for properly inferring the joint distribution of the variables of interest. Although many studies have addressed the case of continuous variables, few studies have focused on treating discrete variables. This paper presents a nonparametric approach to the estimation of joint discrete distributions with bounded support using copulas and Bernstein polynomials. We present an application in real obsessive-compulsive disorder data.

Longitudinal latent variable models given incompletely observed biomarkers and covariates.

In this paper, we analyze a two-level latent variable model for longitudinal data from the National Growth and Health Study where surrogate outcomes or biomarkers and covariates are subject to missingness at any of the levels. A conventional method for efficient handling of missing data is to re-express the desired model as a joint distribution of variables, including the biomarkers, that are subject to missingness conditional on all of the covariates that are completely observed, and estimate the joint model by maximum likelihood, which is then transformed to the desired model. The joint model, however, identifies more parameters than desired, in general. We show that the over-identified joint model produces biased estimation of the latent variable model and describe how to impose constraints on the joint model so that it has a one-to-one correspondence with the desired model for unbiased estimation. The constrained joint model handles missing data efficiently under the assumption of ignorable missing data and is estimated by a modified application of the expectation-maximization algorithm. Copyright © 2016 John Wiley & Sons, Ltd.

General location model with factor analyzer covariance matrix structure and its applications

General location model (GLOM) is a well-known model for analyzing mixed data. In GLOM one decomposes the joint distribution of variables into conditional distribution of continuous variables given categorical outcomes and marginal distribution of categorical variables. The first version of GLOM assumes that the covariance matrices of continuous multivariate distributions across cells, which are obtained by different combination of categorical variables, are equal. In this paper, the GLOMs are considered in both cases of equality and unequality of these covariance matrices. Three covariance structures are used across cells: the same factor analyzer, factor analyzer with unequal specific variances matrices (in the general and parsimonious forms) and factor analyzers with common factor loadings. These structures are used for both modeling covariance structure and for reducing the number of parameters. The maximum likelihood estimates of parameters are computed via the EM algorithm. As an application for these models, we investigate the classification of continuous variables within cells. Based on these models, the classification is done for usual as well as for high dimensional data sets. Finally, for showing the applicability of the proposed models for classification, results from analyzing three real data sets are presented.

Complexity reduction of influence nets using arc removal

The model building of Influence Nets, a special instance of Bayesian belief networks, is a time-consuming and labor-intensive task. No formal process exists that decision makers/system analyst, who are typically not familiar with the underlying theory and assumptions of belief networks, can use to build concise and easy-to-interpret models. In many cases, the developed model is extremely dense, that is, it has a very high link-to-node ratio. The complexity of a network makes the already intractable task of belief updating more difficult. The problem is further intensified in dynamic domains where the structure of the built model is repeated for multiple time-slices. It is, therefore, desirable to do a post-processing of the developed models and to remove arcs having a negligible influence on the variables of interests. The paper applies sensitivity of arc analysis to identify arcs that can be removed from an Influence Net without having a significant impact on its inferencing capability. A metric is suggested to gauge changes in the joint distribution of variables before and after the arc removal process. The results are benchmarked against the KL divergence metric. An empirical study based on several real Influence Nets is conducted to test the performance of the sensitivity of arc analysis in reducing the model complexity of an Influence Net without causing a significant change in its joint probability distribution.

Empirical Information Metrics for Prediction Power and Experiment Planning

In principle, information theory could provide useful metrics for statistical inference. In practice this is impeded by divergent assumptions: Information theory assumes the joint distribution of variables of interest is known, whereas in statistical inference it is hidden and is the goal of inference. To integrate these approaches we note a common theme they share, namely the measurement of prediction power. We generalize this concept as an information metric, subject to several requirements: Calculation of the metric must be objective or model-free; unbiased; convergent; probabilistically bounded; and low in computational complexity. Unfortunately, widely used model selection metrics such as Maximum Likelihood, the Akaike Information Criterion and Bayesian Information Criterion do not necessarily meet all these requirements. We define four distinct empirical information metrics measured via sampling, with explicit Law of Large Numbers convergence guarantees, which meet these requirements: Ie, the empirical information, a measure of average prediction power; Ib, the overfitting bias information, which measures selection bias in the modeling procedure; Ip, the potential information, which measures the total remaining information in the observations not yet discovered by the model; and Im, the model information, which measures the model’s extrapolation prediction power. Finally, we show that Ip + Ie, Ip + Im, and Ie — Im are fixed constants for a given observed dataset (i.e. prediction target), independent of the model, and thus represent a fundamental subdivision of the total information contained in the observations. We discuss the application of these metrics to modeling and experiment planning.

Conditional copulas, association measures and their applications

One way to model a dependence structure is through the copula function which is a mean to capture the dependence structure in the joint distribution of variables. Association measures such as Kendall’s tau or Spearman’s rho can be expressed as functionals of the copula. The dependence structure between two variables can be highly influenced by a covariate, and it is of real interest to know how this dependence structure changes with the value taken by the covariate. This motivates the need for introducing conditional copulas, and the associated conditional Kendall’s tau and Spearman’s rho association measures. After the introduction and motivation of these concepts, two nonparametric estimators for a conditional copula are proposed and discussed. Then nonparametric estimates for the conditional association measures are derived. A key issue is that these measures are now looked at as functions in the covariate. The performances of all estimators are investigated via a simulation study which also includes a data-driven algorithm for choosing the smoothing parameters. The usefulness of the methods is illustrated on two real data examples.

A distance-based rounding strategy for post-imputation ordinal data

Multiple imputation has emerged as a widely used model-based approach in dealing with incomplete data in many application areas. Gaussian and log-linear imputation models are fairly straightforward to implement for continuous and discrete data, respectively. However, in missing data settings which include a mix of continuous and discrete variables, correct specification of the imputation model could be a daunting task owing to the lack of flexible models for the joint distribution of variables of different nature. This complication, along with accessibility to software packages that are capable of carrying out multiple imputation under the assumption of joint multivariate normality, appears to encourage applied researchers for pragmatically treating the discrete variables as continuous for imputation purposes, and subsequently rounding the imputed values to the nearest observed category. In this article, I introduce a distance-based rounding approach for ordinal variables in the presence of continuous ones. The first step of the proposed rounding process is predicated upon creating indicator variables that correspond to the ordinal levels, followed by jointly imputing all variables under the assumption of multivariate normality. The imputed values are then converted to the ordinal scale based on their Euclidean distances to a set of indicators, with minimal distance corresponding to the closest match. I compare the performance of this technique to crude rounding via commonly accepted accuracy and precision measures with simulated data sets.

Joint distributions of arbitrary variables made easy

The successful application of joint time-frequency distributions to problems in time-varying spectral analysis has stimulated considerable interest in distributions of other variables for use when a strict time-frequency analysis is not appropriate. Joint distributions generalise single variable distributions that measure the energy content of some physical quantity in a signal. We propose a new framework for studying joint distributions of variables beyond time-frequency and time-scale. When applicable, our results turn the theory of joint distributions of arbitrary variables into an easy exercise of coordinate transformation. While straightforward, our method can generate many distributions previously attainable only by the general method of Cohen (see Time-Frequency Analysis. Englewood Cliffs, NJ: Prentice-Hall, 1995).

Ecologic studies in epidemiology: concepts, principles, and methods.

An ecologic study focuses on the comparison of groups, rather than individuals; thus, individual-level data are missing on the joint distribution of variables within groups. Variables in an ecologic analysis may be aggregate measures, environmental measures, or global measures. The purpose of an ecologic analysis may be to make biologic inferences about effects on individual risks or to make ecologic inferences about effects on group rates. Ecologic study designs may be classified on two dimensions: (a) whether the primary group is measured (exploratory vs analytic study); and (b) whether subjects are grouped by place (multiple-group study), by time (time-trend study), or by place and time (mixed study). Despite several practical advantages of ecologic studies, there are many methodologic problems that severely limit causal inference, including ecologic and cross-level bias, problems of confounder control, within-group misclassification, lack of adequate data, temporal ambiguity, collinearity, and migration across groups.

Duality relations for certain single server queues

In this paper, relations are given between the joint distribution of several variables in a GI/G/1 queue and the joint distribution of variables associated with the busy cycle in the dual queue, that is in the queue which results from the original when the interarrival times and the service times are interchanged. It is assumed that the primal queue has the preemptive-resume last-come-first-served queue discipline while the dual queue may have any queue discipline which is work conserving. These relations generalize a result given recently for M/G/1 and GI/M/1 queues.

Stock and flow sampling

This letter is concerned with the joint distribution of variables that effect the duration of stay in a given state, for example, unemployment. It deals with the difference in the joint distribution between the population of residents of the state—the stock—and the population of entrants or leavers—the flow, and with the implications of this difference for econometric inferences.

An analysis of statistical methods for comparing obstetric outcomes: Infant health in three samples of school-age pregnancies

Obstetric data on infants to be used in comparative studies should be reported in terms of the joint distribution of the relevant variables, in order that maximum statistical power may be achieved. A ranking test and a scoring method, both of which depend on the joint distribution of variables, are described and applied to the study data. The ranking test and the scoring method are shown to have greater statistical power to detect differences between programs than examination of individual variables by the chi-square test, and the scoring method has the advantages of simplicity and clarity in comparing the outcomes of several different programs. Infants of teen-age mothers who participated in a comprehensive program of medical, social, and education services (YMP) fared better at birth than infants of teen-age mothers who did not participate in such a program (control group). Infants of mothers who participated in a program of special educational and social services with routine medical services could not be shown to fare better than mothers in the control gruop. When the two special programs were compared directly, a statistically significant difference could not be demonstrated, although the program with the greater emphasis on medical services (YMP) showed a tendency in the direction predicted.