Marginal Distribution Of Covariates Research Articles

Covariate-Shift Generalization via Random Sample Weighting

Shifts in the marginal distribution of covariates from training to the test phase, named covariate-shifts, often lead to unstable prediction performance across agnostic testing data, especially under model misspecification. Recent literature on invariant learning attempts to learn an invariant predictor from heterogeneous environments. However, the performance of the learned predictor depends heavily on the availability and quality of provided environments. In this paper, we propose a simple and effective non-parametric method for generating heterogeneous environments via Random Sample Weighting (RSW). Given the training dataset from a single source environment, we randomly generate a set of covariate-determining sample weights and use each weighted training distribution to simulate an environment. We theoretically show that under appropriate conditions, such random sample weighting can produce sufficient heterogeneity to be exploited by common invariance constraints to find the invariant variables for stable prediction under covariate shifts. Extensive experiments on both simulated and real-world datasets clearly validate the effectiveness of our method.

Indirect covariate balance and residual confounding: An applied comparison of propensity score matching and cardinality matching.

Propensity score matching (PSM) is subject to limitations associated with limited degrees of freedom and covariate overlap. Cardinality matching (CM), an optimization algorithm, overcomes these limitations by matching directly on the marginal distribution of covariates. This study compared the performance of PSM and CM. Comparative cohort study of new users of angiotensin-converting enzyme inhibitor (ACEI) and β-blocker monotherapy identified from a large U.S. administrative claims database. One-to-one matching was conducted through PSM using nearest-neighbor matching (caliper=0.15) and CM permitting a maximum standardized mean difference (SMD) of 0, 0.01, 0.05, and 0.10 between comparison groups. Matching covariates included 37 patient demographic and clinical characteristics. Observed covariates included patient demographics, and all observed prior conditions, drug exposures, and procedures. Residual confounding was assessed based on the expected absolute systematic error of negative control outcome experiments. PSM and CM were compared in terms of post-match patient retention, matching and observed covariate balance, and residual confounding within a 10%, 1%, 0.25% and 0.125% sample group. The eligible study population included 182 235 (ACEI: 129363; β-blocker: 56872) patients. CM achieved superior patient retention and matching covariate balance in all analyses. After PSM, 1.6% and 28.2% of matching covariates were imbalanced in the 10% and 0.125% sample groups, respectively. No significant difference in observed covariate balance was observed between matching techniques. CM permitting a maximum SMD <0.05 was associated with improved residual bias as compared to PSM. We recommend CM with more stringent balance criteria as an alternative to PSM when matching on a set of clinically relevant covariates.

Adaptive Bayesian estimation of conditional discrete-continuous distributions with an application to stock market trading activity

We consider Bayesian nonparametric estimation of conditional discrete-continuous distributions. Our model is based on a mixture of normal distributions with covariate dependent mixing probabilities. We use continuous latent variables for modeling the discrete part of the distribution. The marginal distribution of covariates is not modeled. Under anisotropic smoothness conditions on the data generating conditional distribution and a possibly increasing number of the support points for the discrete part of the distribution, we show that the posterior in our model contracts at frequentist adaptive optimal rates up to a log factor. Our results also imply an upper bound on the posterior contraction rate for predictive distributions when the data follow an ergodic Markov process and our model is used for modeling the Markov transition distribution. The proposed model performs well in an application to stock market trading activity.

Counterfactual: An R Package for Counterfactual Analysis

The Counterfactual package implements the estimation and inference methods of Cher nozhukov et al. (2013) for counterfactual analysis. The counterfactual distributions considered are the result of changing either the marginal distribution of covariates related to the outcome variable of interest, or the conditional distribution of the outcome given the covariates. They can be applied to estimate quantile treatment effects and wage decompositions. This paper serves as an introduction to the package and displays basic functionality of the commands contained within.

ADAPTIVE BAYESIAN ESTIMATION OF CONDITIONAL DENSITIES

We consider a nonparametric Bayesian model for conditional densities. The model is a finite mixture of normal distributions with covariate dependent multinomial logit mixing probabilities. A prior for the number of mixture components is specified on positive integers. The marginal distribution of covariates is not modeled. We study asymptotic frequentist behavior of the posterior in this model. Specifically, we show that when the true conditional density has a certain smoothness level, then the posterior contraction rate around the truth is equal up to a log factor to the frequentist minimax rate of estimation. An extension to the case when the covariate space is unbounded is also established. As our result holds without a priori knowledge of the smoothness level of the true density, the established posterior contraction rates are adaptive. Moreover, we show that the rate is not affected by inclusion of irrelevant covariates in the model. In Monte Carlo simulations, a version of the model compares favorably to a cross-validated kernel conditional density estimator.

A paradox: more measurement error can lead to more efficient estimates

We investigated the conditions under which the information matrix, conditional on covariates, and the unconditional version, integrated over marginal distribution of covariates, are increasing functions of the measurement error variance, and when the conditional and unconditional asymptotic variances of the maximum likelihood estimate are decreasing functions of the measurement error variance. We say that a paradox occurs when one can decrease the variance of the maximum likelihood estimate by increasing measurement error. Two covariate measurement error models were considered: the Berk-son and the classical measurement error models, with continuous and binary dependent variables, The measurement error variance was assumed known. We found that in the linear model with Berkson eovariate measurement error, the paradox can occur when the model variance is known and quite large. With binary data and the Berkson eovariate measurement error, the paradox is likely to occur for rare events. The paradox based upon the unconditional variance, calculated as the inverse of the unconditional information matrix, does not occur with the classical measurement error model. However, if does occur for the conditional variance, as calculated by the inverse of the observed information matrix, in some circumstances. Empirical evidence for this is illustrated using data from the Nurses' Health Study.