Conditional Independence Restrictions Research Articles

A Generalized Non-Parametric Instrumental Variable-Control Function Approach to Estimation in Nonlinear Settings

Abstract When the endogenous variables enter non-parametrically into the regression equation standard linear instrumental variables approaches fail. Two existing solutions are the non-parametric instrumental variables (NPIVs) estimators, which are based on a set of conditional moment restrictions (CMRs), and the control function (CF) estimators, which use conditional mean independence (CMI) restrictions. Our first contribution is to show that – similar to CMI – the CMR place shape restrictions on the conditional expectation of the error given the instruments and endogenous variables that are sufficient for identification, and we call our new estimator based on these restrictions the CMR-CF estimator. Our second contribution is to develop an estimator for non-linear and non-parametric settings that can combine both CMR and CMI restrictions, which cannot be done in either the NPIV nor the non-parametric CF setting. This new “Generalized CMR-CF” uses both CMR and CMI restrictions together by allowing the conditional expectation of the structural error to depend on both instruments and control variables. When sieves are used to approximate both the structural function and the CF our estimator reduces to a series of least squares regressions. Our Monte Carlos illustrate that our new estimator performs well across several economic settings.

Edge Exclusion Tests for Graphical Model Selection: Complex Gaussian Vectors and Time Series

We consider the problem of inferring the conditional independence graph (CIG) of a stationary multivariate Gaussian time series. A $p$ -variate Gaussian time series graphical model associated with an undirected graph with $p$ vertices is defined as the family of time series that obey the conditional independence restrictions implied by the edge set of the graph. In some existing methods, partial coherence has been used as a test statistic for graphical model selection. To test inclusion/exclusion of a given edge in the graph, the test is applied at distinct frequencies, requiring multiple tests and leading to a loss in test power. The nonparametric method of Matsuda uses the Kullback-Leibler divergence measure to define a test statistic which does not need multiple testing to test between two competing models. In this paper we propose a generalized likelihood ratio test (GLRT) based edge exclusion test statistic that also does not need multiple testing. It is computationally significantly faster than the method of Matsuda, and simulations show that we achieve comparable power levels. Our proposed approach is based on a novel formulation of a GLRT based edge exclusion test for $p$ -variate complex Gaussian graphical models (CGGMs); this result is also of independent interest. The computational complexity of the proposed statistic is $\mathcal {O}(p^3)$ compared to $\mathcal {O}(p^5)$ for the existing result. We also apply our time series graphical model selection method to a foreign exchange data set consisting of monthly trends of foreign exchange rates of 10 countries.

Edge Exclusion Tests for Improper Complex Gaussian Graphical Model Selection

We consider the problem of inferring the conditional independence graph (CIG) of improper complex-valued Gaussian vectors. A $p$ -variate improper complex Gaussian graphical model associated with an undirected graph with $p$ vertices is defined as the family of improper complex Gaussian distributions that obey the conditional independence restrictions implied by the edge set of the graph. For real random vectors, considerable body of paper exists where one first tests for exclusion of each edge from the saturated model, and then infers the CIG. Prior work on proper complex Gaussian graphical models is sparse, while that on improper complex Gaussian graphical models is non-existent. In this paper, we propose and analyze a generalized likelihood ratio test (GLRT) based edge exclusion test statistic for improper complex Gaussian graphical models. The null distribution of the test statistic is specified explicitly to allow analytical calculation of the test threshold. An alternative computationally fast version of the GLRT statistic is also derived. Simulation examples are presented to illustrate the proposed statistic.

Partial identification by extending subdistributions

I show that sharp identified sets in a large class of econometric models can be characterized by solving linear systems of equations. These linear systems determine whether, for a given value of a parameter of interest, there exists an admissible joint distribution of unobservables that can generate the distribution of the observed variables. The joint distribution of unobservables is not required to satisfy any parametric restrictions, but can (if desired) be assumed to satisfy a variety of location, shape, and/or conditional independence restrictions. To prove sharpness of the characterization, I generalize a classic result in copula theory concerning the extendibility of subcopulas to show that related objects—termed subdistributions—can be extended to proper distribution functions. I describe this characterization argument as partial identification by extending subdistributions, or PIES. One particularly attractive feature of PIES is that it focuses directly on the sharp identified set for a parameter of interest, such as an average treatment effect, without needing to construct the identified set for the entire model. I apply PIES to univariate and bivariate binary response models. A notable product of the analysis is a method for characterizing the sharp identified set for the average treatment effect in Manski's (1975, 1985, 1988) semiparametric binary response model. Partial identification maximum score bivariate probit copulas linear programming discrete choice semiparametric endogeneity C14 C20 C51

Efficient estimation in models with independence restrictions

Unconditional and conditional independence restrictions are used in many econometric models to identify their parameters. However, there are few results about efficient estimation procedures for finite-dimensional parameters under these independence restrictions. This paper computes the efficiency bound for finite-dimensional parameters under independence restrictions, and proposes an estimator that is consistent, asymptotically normal and which achieves the efficiency bound. The estimator is based on a growing number of zero-covariance conditions that are asymptotically equivalent to the independence restriction. The results are illustrated with examples, including an instrumental variables regression model and partially linear regression models. A small Monte Carlo study is performed to investigate the estimator’s small sample properties and to quantify the efficiency gains that can be made by using the proposed efficient estimator.

Partial Identification by Extending Subdistributions

I show that sharp identified sets in a large class of econometric models can be characterized by solving linear systems of equations. These linear systems determine whether, for a given value of a parameter of interest, there exists an admissible joint distribution of unobservables that can generate the distribution of the observed variables. The parameter of interest can be a structural function, but it can also be a more complicated feature of the model primitives, such as an average treatment effect. The joint distribution of unobservables is not required to satisfy any parametric restrictions, but can (if desired) be assumed to satisfy a variety of location, shape and/or conditional independence restrictions. To prove that this characterization is sharp, I generalize a classic lemma in copula theory concerning the extendibility of subcopulas to show that related objects, termed subdistributions, can be extended to proper distribution functions. This result is then used to reduce the characterization of the identified set to the determination of the existence or non-existence of suitably-constrained subdistributions, which in turn is often equivalent to solving a linear system of equations. I describe this argument as partial identification by extending subdistributions, or PIES. I apply PIES to an ordered discrete response model and a two-sector Roy model. One product of the first application is a tractable characterization of the sharp identified set for the average treatment effect in the semiparametric binary response model considered by Manski (1975, 1985, 1988).

Efficient Estimation in Models with Independence Restrictions

Unconditional or conditional independence restrictions are used in many econometric models to identify their parameters. However, there are few results about efficient estimation procedures for finite-dimensional parameters under these independence restrictions. This paper computes the efficiency bound for finite-dimensional parameters under independence restrictions, and proposes an estimator that is consistent, asymptotically normal and which achieves the efficiency bound. The estimator is based on a growing number of zero-covariance conditions that are asymptotically equivalent to the independence restriction. The results are illustrated with examples, including an instrumental variables regression model and partially linear regression models. A small Monte-Carlo study is performed to investigate the estimator’s small sample properties and to quantify the efficiency gains that can be made by using the proposed efficient estimator.

A class of smooth models satisfying marginal and context specific conditional independencies

We study a class of conditional independence models for discrete data with the property that one or more log-linear interactions are defined within two different marginal distributions and then constrained to 0; all the conditional independence models which are known to be non-smooth belong to this class. We introduce a new marginal log-linear parameterization and show that smoothness may be restored by restricting one or more independence statements to hold conditionally to a restricted subset of the configurations of the conditioning variables. Our results are based on a specific reconstruction algorithm from log-linear parameters to probabilities and fixed point theory. Several examples are examined and a general rule for determining the implied conditional independence restrictions is outlined.

Testing Conditional Independence Restrictions

We propose a nonparametric test of the hypothesis of conditional independence between variables of interest based on a generalization of the empirical distribution function. This hypothesis is of interest both for model specification purposes, parametric and semiparametric, and for nonmodel-based testing of economic hypotheses. We allow for both discrete variables and estimated parameters. The asymptotic null distribution of the test statistic is a functional of a Gaussian process. A bootstrap procedure is proposed for calculating the critical values. Our test has power against alternatives at distance n −1/2 from the null; this result holding independently of dimension. Monte Carlo simulations provide evidence on size and power.

Marginal log-linear parameterization of conditional independence models

Models defined by a set of conditional independence restrictions play an important role in statistical theory and applications, especially, but not only, in graphical modelling. In this paper we identify a subclass of these consisting of hierarchical marginal log-linear models, as defined by Bergsma & Rudas (2002a). Such models are smooth, which implies the applicability of standard asymptotic theory and simplifies interpretation. Furthermore, we give a marginal log-linear parameterization and a minimal specification of the models in the subclass, which implies the applicability of standard methods to compute maximum likelihood estimates and simplifies the calculation of the degrees of freedom of chi-squared statistics to test goodness-of-fit. The utility of the results is illustrated by applying them to block-recursive Markov models associated with chain graphs.

High-dimensional graphs and variable selection with the Lasso

The pattern of zero entries in the inverse covariance matrix of a multivariate normal distribution corresponds to conditional independence restrictions between variables. Covariance selection aims at estimating those structural zeros from data. We show that neighborhood selection with the Lasso is a computationally attractive alternative to standard covariance selection for sparse high-dimensional graphs. Neighborhood selection estimates the conditional independence restrictions separately for each node in the graph and is hence equivalent to variable selection for Gaussian linear models. We show that the proposed neighborhood selection scheme is consistent for sparse high-dimensional graphs. Consistency hinges on the choice of the penalty parameter. The oracle value for optimal prediction does not lead to a consistent neighborhood estimate. Controlling instead the probability of falsely joining some distinct connectivity components of the graph, consistent estimation for sparse graphs is achieved (with exponential rates), even when the number of variables grows as the number of observations raised to an arbitrary power.

Joint Response Graphs and Separation Induced by Triangular Systems

SummaryWe consider joint probability distributions generated recursively in terms of univariate conditional distributions satisfying conditional independence restrictions. The independences are captured by missing edges in a directed graph. A matrix form of such a graph, called the generating edge matrix, is triangular so the distributions that are generated over such graphs are called triangular systems. We study consequences of triangular systems after grouping or reordering of the variables for analyses as chain graph models, i.e. for alternative recursive factorizations of the given density using joint conditional distributions. For this we introduce families of linear triangular equations which do not require assumptions of distributional form. The strength of the associations that are implied by such linear families for chain graph models is derived. The edge matrices of chain graphs that are implied by any triangular system are obtained by appropriately transforming the generating edge matrix. It is shown how induced independences and dependences can be studied by graphs, by edge matrix calculations and via the properties of densities. Some ways of using the results are illustrated.

Linear Regression Models under Conditional Independence Restrictions

AbstractMaximum likelihood estimation is investigated in the context of linear regression models under partial independence restrictions. These restrictions aim to assume a kind of completeness of a set of predictors Z in the sense that they are sufficient to explain the dependencies between an outcome Y and predictors X: ℒ(Y|Z, X) = ℒ(Y|Z), where ℒ(·|·) stands for the conditional distribution. From a practical point of view, the former model is particularly interesting in a double sampling scheme where Y and Z are measured together on a first sample and Z and X on a second separate sample. In that case, estimation procedures are close to those developed in the study of double‐regression by Engel & Walstra (1991) and Causeur & Dhorne (1998). Properties of the estimators are derived in a small sample framework and in an asymptotic one, and the procedure is illustrated by an example from the food industry context.

Labelled Graphical Models

AbstractA class of log‐linear models, referred to as labelled graphical models (LGMs), is introduced for multinomial distributions. These models generalize graphical models (GMs) by employing partial conditional independence restrictions which are valid only in subsets of an outcome space. Theoretical results concerning model identifiability, decomposability and estimation are derived. A decision theoretical framework and a search algorithm for the identification of plausible models are described. Real data sets are used to illustrate that LGMs may provide a simpler interpretation of a dependence structure than GMs.

Invariant normal models with recursive graphical Markov structure

An extension of the class of GS-LCI normal models introduced by Andersson and Madsen is defined and studied. The models are defined in terms of symmetry restrictions given by a finite group and conditional independence restrictions given by an acyclic directed graph. Maximum likelihood estimation of the parameters in the models is discussed.

Lattice conditional independence models for seemingly unrelated regressions

Seemingly unrelated regressions (SUR) models appear frequently in econometrics and in the analyses of repeated measures designs and longitudinal data. It is known that iterative algorithms are generally required to obtain the MLEs of the regression parameters. Under a minimal set of lattice conditional independence (LCI) restrictions imposed on the covariance structure, however, closed-form MLEs can be obtained by standard linear regression techniques (Andersson and Perlman, 1993, 1994, 1998). In this paper, simulation is used to study the efficiency of these LCI model-based estimators. We also propose two possible improvements of the usual two-stage estimators for the regression parameters.

Symmetry and lattice conditional independence in a multivariate normal distribution

A class of multivariate normal models with symmetry restrictions given by a finite group and conditional independence restrictions given by a finite distributive lattice is defined and studied. The statistical properties of these models including maximum likelihood inference, invariance and hypothesis testing are discussed.

Hyper Markov Laws in the Statistical Analysis of Decomposable Graphical Models

This paper introduces and investigates the notion of a hyper Markov law, which is a probability distribution over the set of probability measures on a multivariate space that (i) is concentrated on the set of Markov probabilities over some decomposable graph, and (ii) satisfies certain conditional independence restrictions related to that graph. A stronger version of this hyper Markov property is also studied. Our analysis starts by reconsidering the properties of Markov probabilities, using an abstract approach which thereafter proves equally applicable to the hyper Markov case. Next, it is shown constructively that hyper Markov laws exist, that they appear as sampling distributions of maximum likelihood estimators in decomposable graphical models, and also that they form natural conjugate prior distributions for a Bayesian analysis of these models. As examples we construct a range of specific hyper Markov laws, including the hyper multinomial, hyper Dirichlet and the hyper Wishart and inverse Wishart laws. These laws occur naturally in connection with the analysis of decomposable log-linear and covariance selection models.

Lattice-ordered conditional independence models for missing data

Statistical inference for the parameters of a multivariate normal distribution N p (μ, Σ) based on a sample with missing observations is straightforward when the missing data pattern is monotone (= nested), reducing to the analysis of several normal linear regression models by step-wise conditioning. When the missing data pattern is non-monotone, however, such analysis is impossible. It is shown here that every missing data pattern naturally determines a set of lattice-ordered conditional independence restrictions which, when imposed upon the unknown covariance matrix Σ, yields a factorization of the joint likelihood function as a product of (conditional) likelihood functions of normal linear regression models just as in the monotone case. From this factorization the maximum likelihood estimators of μ and Σ (under the conditional independence restrictions) can be explicitly derived.

Decomposable Models: A New Look at Interdependence and Dependence Structures in Psychological Research.

Decomposable models represent interdependence structures for observable variables. Each model is fully characterized by a set of conditional independence restrictions, and can be visualized with an undirected as well as a special type of a directed graph. As a consequence each decomposable model can be interpreted either in terms of interdependencies only or as a particular kind of dependence structure, as a recursive system or path analysis model. Under the assumption of normally distributed variables, decomposable models determine the structure of correlation matrices, and maximum-likelihood estimates of these can be calculated with the help of ordinary least squares estimation. Using several examples from psychological research, we discuss the interpretation of decomposable models. Furthermore, it is demonstrated how recursive dependence structures can be specified with the help of decomposable models in a hypothesis generating (exploratory) as well as in a hypothesis testing (confirmatory) manner.