Oregon Health Insurance Experiment Research Articles

Machine Learning Detects Heterogeneous Effects of Medicaid Coverage on Depression.

In 2008, Oregon expanded its Medicaid program using a lottery, creating a rare opportunity to study the effects of Medicaid coverage using a randomized controlled design (Oregon Health Insurance Experiment). Analysis showed that Medicaid coverage lowered the risk of depression. However, this effect may vary between individuals, and the identification of individuals likely to benefit the most has the potential to improve the effectiveness and efficiency of the Medicaid program. By applying the machine learning causal forest to data from this experiment, we found substantial heterogeneity in the effect of Medicaid coverage on depression; individuals with high predicted benefit were older and had more physical or mental health conditions at baseline. Expanding coverage to individuals with high predicted benefit generated greater reduction in depression prevalence than expanding to all eligible individuals (21.5 vs. 8.8 percentage point reduction; adjusted difference [95%CI], +12.7 [+4.6,+20.8]; P=0.003), at substantially lower cost per case prevented ($16,627 vs. $36,048; adjusted difference [95%CI], -$18,598 [-$156,953,-$3,120]; P=0.04). Medicaid coverage reduces depression substantially more in a subset of the population than others, in ways that are predictable in advance. Targeting coverage on those most likely to benefit could improve the effectiveness and efficiency of insurance expansion.

Heterogeneity within the Oregon Health Insurance Experiment: An application of causal forests.

Existing evidence regarding the effects of Medicaid expansion, largely focused on aggregate effects, suggests health insurance impacts some health, healthcare utilization, and financial hardship outcomes. In this study we apply causal forest and instrumental forest methods to data from the Oregon Health Insurance Experiment (OHIE), to explore heterogeneity in the uptake of health insurance, and in the effects of (a) lottery selection and (b) health insurance on a range of health-related outcomes. The findings of this study suggest that the impact of winning the lottery on the health insurance uptake varies among different subgroups based on age and race. In addition, the results generally coincide with findings in the literature regarding the overall effects: lottery selection (and insurance) reduces out-of-pocket spending, increases physician visits and drug prescriptions, with little (short-term) impact on the number of emergency department visits and hospital admissions. Despite this, we detect quite weak evidence of heterogeneity in the effects of the lottery and of health insurance across the outcomes considered.

Efficient and Robust Estimation of the Generalized LATE Model

This article studies the estimation of causal parameters in the generalized local average treatment effect (GLATE) model, which expands upon the traditional LATE model to include multivalued treatments. We derive the efficient influence function (EIF) and the semiparametric efficiency bound (SPEB) for two types of causal parameters: the local average structural function (LASF) and the local average structural function for the treated (LASFT). The moment conditions generated by the EIF satisfy two robustness properties: double robustness and Neyman orthogonality. Based on the robust moment conditions, we propose the double/debiased machine learning (DML) estimator for estimating the LASF. The DML estimator is well-suited for high dimensional settings. We also propose null-restricted inference methods that are robust against weak identification issues. As an empirical application of these methods, we examine the potential health outcome across different types of health insurance plans using data from the Oregon Health Insurance Experiment.

Sharp bounds in the latent index selection model

A fundamental question underlying the literature on partial identification is: what can we learn about parameters that are relevant for policy but not necessarily point-identified by the exogenous variation we observe? This paper provides an answer in terms of sharp, analytic characterizations and bounds for an important class of policy-relevant treatment effects, consisting of marginal treatment effects and linear functionals thereof, in the latent index selection model as formalized in Vytlacil (2002). The sharp bounds use the full content of identified marginal distributions, and analytic derivations rely on the theory of stochastic orders. The proposed methods also make it possible to sharply incorporate new auxiliary assumptions on distributions into the latent index selection framework. Empirically, I apply the methods to study the effects of Medicaid on emergency room utilization in the Oregon Health Insurance Experiment, showing that the predictions from extrapolations based on a distribution assumption (rank similarity) differ substantively and consistently from existing extrapolations based on a parametric mean assumption (linearity). This underscores the value of utilizing the model’s full empirical content in combination with auxiliary assumptions.

Reconciling Seemingly Contradictory Results from the Oregon Health Insurance Experiment and the Massachusetts Health Reform

Abstract A headline result from the Oregon Health Insurance Experiment is that emergency room (ER) utilization increased. A seemingly contradictory result from the Massachusetts health reform is that ER utilization decreased. I reconcile both results by identifying treatment effect heterogeneity within the Oregon experiment and extrapolating it to Massachusetts. Even though Oregon compliers increased their ER utilization, they were adversely selected relative to Oregon never takers, who would have decreased their ER utilization. Massachusetts expanded coverage from a higher level to healthier compliers. Therefore, Massachusetts compliers are comparable to a subset of Oregon never takers, which can reconcile the results.

Testing attributable effects hypotheses with an application to the Oregon Health Insurance Experiment

Testing attributable effects hypotheses with an application to the Oregon Health Insurance Experiment

Out of the Woodwork: Enrollment Spillovers in the Oregon Health Insurance Experiment.

We analyze the impact of expanded adult Medicaid eligibility on the enrollment of already-eligible children. We analyze the 2008 Oregon Medicaid lottery, in which some low-income uninsured adults were randomly selected to be allowed to apply for Medicaid. Children in these households were eligible for Medicaid irrespective of the lottery outcome. We estimate statistically significant but transitory impacts of adult lottery selection on child Medicaid enrollment: at three months after the lottery, for every 9 adults who enrolled in Medicaid due to winning the lottery, one additional child also enrolled. Our results shed light on the existence, magnitude, and nature of so-called "woodwork effects".

Detecting heterogeneous treatment effects with instrumental variables and application to the Oregon health insurance experiment

There is an increasing interest in estimating heterogeneity in causal effects in randomized and observational studies. However, little research has been conducted to understand effect heterogeneity in an instrumental variables study. In this work we present a method to estimate heterogeneous causal effects using an instrumental variable with matching. The method has two parts. The first part uses subject-matter knowledge and interpretable machine-learning techniques, such as classification and regression trees, to discover potential effect modifiers. The second part uses closed testing to test for statistical significance of each effect modifier while strongly controlling the familywise error rate. We apply this method on the Oregon Health Insurance Experiment, estimating the effect of Medicaid on the number of days an individual’s health does not impede their usual activities by using a randomized lottery as an instrument. Our method revealed Medicaid’s effect was most impactful among older, English-speaking, non-Asian males and younger, English-speaking individuals with, at most, a high school diploma or General Educational Development.

Medicaid, Health, and the Moderating Role of Neighborhood Characteristics.

Neighborhood characteristics may moderate the effects of Medicaid coverage on health outcomes. Identifying this interplay can inform the design, targeting, and implementation of health policy. We combine data from the Oregon Health Insurance Experiment, which randomized access to Medicaid, with rich new data on multiple domains of neighborhood characteristics to assess the interaction between the local environment and the effect of insurance on health. Overall, we do not find that neighborhood characteristics substantially affect the relationship between gaining insurance and health outcomes, suggesting that Medicaid expansions are similarly effective across neighborhoods. This analysis highlights the complex relationship between health insurance coverage, neighborhood characteristics, and health.

Who Increases Emergency Department Use? New Insights from the Oregon Health Insurance Experiment

We provide new insights regarding the finding that Medicaid increased emergency department (ED) use from the Oregon experiment. We find meaningful heterogeneous impacts of Medicaid on ED use using causal machine learning methods. The treatment effect distribution is widely dispersed, and the average effect is not representative of most individualized treatment effects. A small group - about 14% of participants - in the right tail of the distribution drives the overall effect. We identify priority groups with economically significant increases in ED usage based on demographics and prior utilization. Intensive margin effects are an important driver of increases in ED utilization.

Who Increases Emergency Department Use? New Insights from the Oregon Health Insurance Experiment

Who Increases Emergency Department Use? New Insights from the Oregon Health Insurance Experiment

Who Increases Emergency Department Use? New Insights from the Oregon Health Insurance Experiment

Who Increases Emergency Department Use? New Insights from the Oregon Health Insurance Experiment

Medicaid, Earnings, and Heterogeneous Treatment Effects: Evidence from the Oregon Health Insurance Experiment

Abstract The Medicaid and labor supply empirical literature offers competing conclusions of zero effects and significant reductions in earnings. However, zero effects are only theoretically consistent with the earnings distribution’s extremes. Medicaid participants with positive pre-treatment labor supply should unequivocally decrease earnings. This paper clarifies the literature’s ambiguity by combining quantile regression with data from the Oregon Health Insurance Experiment. The distributional impacts imply that zero effects are not universally representative of Medicaid households. The annual earnings impact of Medicaid participation ranges between increases of $1400 to deceases of $3120 for single adults. Pre-existing mental illness or health constraints on work account for counterintuitive positive earnings impacts. By demonstrating that sample compositional differences determine whether Medicaid’s labor supply impact is zero or negative, this paper offers a reconciliation to the range of existing estimates in the empirical literature.

Decreased Emergency Department Use Following Medicaid Expansion: Evidence from Oregon's Health Insurance Experiment

ABSTRACT In this study, we identify the relationship between Medicaid expansion and emergency department use based on the data from Oregon Health Program. Among the relevant studies, Taubman et al. (2014) provide the positive relationship between two objects with a rather limited approach that leads to the under-identification problem. To avoid the problem, we incorporate a copula regression analysis with an additional endogenous variable. As a result, we find that Medicaid expansion leads to a decrease in emergency department use, which is the opposite result of Taubman et al. (2014).

Estimating population average treatment effects from experiments with noncompliance

Abstract Randomized control trials (RCTs) are the gold standard for estimating causal effects, but often use samples that are non-representative of the actual population of interest. We propose a reweighting method for estimating population average treatment effects in settings with noncompliance. Simulations show the proposed compliance-adjusted population estimator outperforms its unadjusted counterpart when compliance is relatively low and can be predicted by observed covariates. We apply the method to evaluate the effect of Medicaid coverage on health care use for a target population of adults who may benefit from expansions to the Medicaid program. We draw RCT data from the Oregon Health Insurance Experiment, where less than one-third of those randomly selected to receive Medicaid benefits actually enrolled.

Connecting Neighborhood Characteristics and Clinical Health Outcomes: Novel Data from the Oregon Health Insurance Experiment

Although policy often emphasizes health care expansions to improve population health, neighborhood characteristics may also influence outcomes. Documenting the association between neighborhood features and health poses several empirical challenges, including the potential for bias from relying on self-reported outcomes, selection bias from using health care data, and contextual limitations due to a methodological focus on one element of neighborhood attributes. In this exploratory analysis, we introduce rich data linking individual clinical health outcomes from a low-income community sample with primary data on a variety of neighborhood characteristics, and investigate the relationships between multiple aspects of neighborhoods and health. We find that neighborhoods have widely varying combinations of attributes and that residents of areas with lower socioeconomic deprivation, greater grocery store availability, and more active living characteristics were in better health than individuals living outside of these areas. Although this analysis does not establish causal pathways, it provides a foundation for future analyses to examine how neighborhoods interact with health policy and health care access to improve population health.

Out of the Woodwork: Enrollment Spillovers in the Oregon Health Insurance Experiment

We analyze the impact of expanded adult Medicaid eligibility on the Medicaid enrollment of already-eligible children. To do so, we exploit the 2008 Oregon Medicaid lottery, in which some low-income uninsured adults were randomly selected for the chance to apply for Medicaid. Children in these households were eligible for Medicaid irrespective of whether the household won the lottery. We estimate statistically significant but transitory impacts of adult lottery selection on children’s Medicaid enrollment: for every 9 adults who enroll in Medicaid due to the lottery, one additional child also enrolls at the same time. Our results shed light on the existence, magnitude, and nature of so-called “woodwork effects.

The Value of Medicaid: Interpreting Results from the Oregon Health Insurance Experiment.

We develop a set of frameworks for welfare analysis of Medicaid and apply them to the Oregon Health Insurance Experiment, a Medicaid expansion for low-income, uninsured adults that occurred via random assignment. Across different approaches, we estimate recipient willingness to pay for Medicaid between $0.5 and $1.2 per dollar of the resource cost of providing Medicaid; estimates of the expected transfer Medicaid provides to recipients are relatively stable across approaches, but estimates of its additional value from risk protection are more variable. We also estimate that the resource cost of providing Medicaid to an additional recipient is only 40% of Medicaid's total cost; 60% of Medicaid spending is a transfer to providers of uncompensated care for the low-income uninsured.

Examining the Prevalence of Adverse Childhood Experiences and Associated Cardiovascular Disease Risk Factors Among Low-Income Uninsured Adults.

Adverse childhood experiences (ACEs) are linked to poor adult health outcomes, including cardiovascular disease. However, little is known about its prevalence, specifically in low-income populations. The objective of this study was to estimate the extent of ACEs in a low-income, nonclinical, uninsured adult population and assess the relationship between ACEs and cardiovascular disease risk factors. This study leverages the OHIE's (Oregon Health Insurance Experiment) study population, uninsured adults who were randomly selected to apply for Medicaid, and data collected through in-person health screenings. We objectively measured obesity, cholesterol, blood pressure, and blood sugar. Smoking, physical activity, and history of chronic disease were self-reported. Independent variables were the 10-item ACEs questions covering neglect, abuse, and household dysfunction. The sample consisted of 12 229 low-income, nonelderly uninsured adults who participated in the OHIE health screenings from 2009 to 2010. A total of 5929 (48%) returned a follow-up survey reporting ACEs in 2012. ACEs were more prevalent in low-income adults compared with previous estimates in a general clinical population, with notably high rates of emotional abuse, emotional neglect, and household dysfunction. ACEs were statistically associated with higher rates of obesity, smoking, and physical inactivity, but not high cholesterol or diabetes mellitus. We detected a strong relationship between ACEs and a self-reported history of a hypertension diagnosis but no statistically significant differences in being hypertensive. This study design allowed us to assess the prevalence of ACEs among uninsured low-income adults and the association between ACEs and clinical indicators of cardiovascular disease risk that are difficult to ordinarily observe. Low-income adults have high rates of ACEs than previous prevalence estimates and ACEs were associated with higher rates of multiple cardiovascular disease risk factors. As states continue to expand Medicaid to the previously uninsured, providers may want to consider incorporating trauma-based approaches to care delivery.

Testing the Validity of the Single Interrupted Time Series Design

Given the complex relationships between patients’ demographics, underlying health needs, and outcomes, establishing the causal effects of health policy and delivery interventions on health outcomes is often empirically challenging. The single interrupted time series (SITS) design has become a popular evaluation method in contexts where a randomized controlled trial is not feasible. In this paper, we formalize the structure and assumptions underlying the single ITS design and show that it is significantly more vulnerable to confounding than is often acknowledged and, as a result, can produce misleading results. We illustrate this empirically using the Oregon Health Insurance Experiment, showing that an evaluation using a single interrupted time series design instead of the randomized controlled trial would have produced large and statistically significant results of the wrong sign. We discuss the pitfalls of the SITS design, and suggest circumstances in which it is and is not likely to be reliable.