Counterfactual Case Research Articles

Effectiveness of the enterovirus A71 vaccine on hand, foot, and mouth disease: a real-world study in China

ObjectivesFor the prevention of hand, foot, and mouth disease (HFMD), enterovirus A71 (EV-A71) vaccines have been used in China since 2016. To better inform vaccination strategies, we assess the real-world effectiveness of EV-A71 vaccination in China. MethodsThe analysis was based on surveillance data of HFMD caused by EV-A71 in children under the age of 5 in China, along with meteorological and demographic data. The seasonal autoregressive integrated moving average model and the interrupted time series analysis were used to estimate the effectiveness of the EV-A71 vaccination on the EV-A71 HFMD incidence and to predict the counterfactual cases with no EV-A71 vaccine. ResultsBetween 2010 and 2018, 6 712 613 cases of HFMD caused by EV-A71 were reported in children under 5 years old in 260 Chinese cities. During 2017–2018, the EV-A71 vaccination was associated with a reduction in EV-A71 HFMD incidence, with a relative risk of 0.83 (95% CI, 0.81–0.86), and an estimated reduction of 297 946 (95% CI, 250 534–346 658) cases. However, this association varied across cities (I2 = 85.6%, p < 0.001) and the effectiveness of the EV-A71 vaccination decreased as population density increased. Higher vaccination coverage was associated with greater effectiveness of the EV-A71 vaccination and an earlier point in EV-A71 case reduction. Specifically, when the vaccination coverage exceeded ∼20%, the relative risk was rapidly reduced to below 0.71 (95% CI, 0.69–0.72). DiscussionOur study demonstrated that the EV-A71 vaccination was associated with a reduction in the incidence of EV-A71 HFMD, but the association varied with regions and was influenced by vaccination coverage and population density. To optimize EV-A71 HFMD prevention, increasing vaccination coverage (>20%) is recommended for children under 5 years old.

Microscopic traffic models, accidents, and insurance losses

AbstractThe paper develops a methodology to enable microscopic models of transportation systems to be accessible for a statistical study of traffic accidents. Our approach is intended to permit an understanding not only of historical losses but also of incidents that may occur in altered, potential future systems. Through such a counterfactual analysis, it is possible, from an insurance, but also from an engineering perspective, to assess the impact of changes in the design of vehicles and transport systems in terms of their impact on road safety and functionality.Structurally, we characterize the total loss distribution approximatively as a mean-variance mixture. This also yields valuation procedures that can be used instead of Monte Carlo simulation. Specifically, we construct an implementation based on the open-source traffic simulator SUMO and illustrate the potential of the approach in counterfactual case studies.

Potential impacts of transportation infrastructure improvements to maize and cassava supply chains in Zambia

Agricultural supply chains play a crucial role in supporting food security in Africa. However, high-resolution supply chain information is often not available, which hinders our ability to determine which interventions in food supply chains would most enhance food security. In this study, we develop a high-resolution supply chain model for essential staple crops in Zambia, aiming to estimate how improvements in transportation infrastructure would impact food security. Specifically, we simulate district-level monthly consumption, trade flows, and storage for maize and cassava in Zambia. We then conduct a counterfactual case study with low transportation costs, discovering that reducing transaction costs leads to higher aggregate net agricultural revenue and aggregate net expenditure. These results indicate that transportation investments are more beneficial to suppliers than to consumers, with implications for household food security in smallholder agriculture. Our study highlights the potential for infrastructure investments to improve food security.

Estimated number of reported vaccine-preventable disease cases averted following the introduction of routine vaccination programs in Sweden, 1910-2019.

Routine childhood vaccination programs have had enormous positive public health impacts worldwide. However, in some areas, these benefits may be impeded by vaccine hesitancy and undervaccination. We estimated the number of reported cases of measles, pertussis, mumps and poliomyelitis averted in Sweden after the introduction of routine childhood vaccination programs. We used annual national data on population size and the number of reported cases of measles (1911-2019), pertussis (1911-2019), mumps (1914-2019) and poliomyelitis (1910-2019) for Sweden. For each disease, we calculated the median and 95% confidence interval of the annual pre-vaccination incidence to estimate the number of counterfactual cases; that is, the estimated number of cases that would have been observed in the post-vaccination period had no vaccine been introduced (median incidence × average annual population). For the post-vaccination periods, we calculated reported cases averted and assumed all decreases were due to vaccines. In total, for all four diseases combined, over 2.1 million cases were reported over the respective surveillance periods. Since the introduction of vaccinations, we estimate that over 1.5 million reported cases of these four diseases combined have been averted: measles (633 091), pertussis (608 670), mumps (262 951) and poliomyelitis (58 240). However, due to underreporting, especially during pre-vaccination years, these are likely underestimates. Since the introduction of these routine childhood vaccination programs in Sweden, a substantial number of reported cases of vaccine-preventable diseases have been averted. Vigilance against both failure to vaccinate and undervaccination is necessary to prevent future increases of these vaccine-preventable diseases.

Sectoral Shocks and the Role of Market Integration: The Case of Natural Gas

We quantify how market integration affects the economic propagation of sectoral supply shocks at the example of a Russian gas shutoff to the European Union. An open-economy, multisector general equilibrium model suggests that the adverse output impact on the European Union shrinks fourfold if integration with the global liquified natural gas (LNG) market is considered. On the flip side, other LNG importers also see adverse output effects. Due to nonlinearities, the combined total economic damage is less than half with integration, compared to the counterfactual case without. Governments should foster further integration to make economies more resilient to supply shocks.

Institutions for effective climate policymaking: Lessons from the case of the United Kingdom

The United Kingdom (UK) is home to one of the most ambitious climate policy regimes in the world, centred around the 2008 Climate Change Act (CCA), the first strategic climate legislation of its kind. Building on prior studies of the CCA while highlighting Germany as a counterfactual case study, we demonstrate the significant positive role that strategic framework legislation can play in improving climate policy integration and coherence. We further show that important new institutions can be established under the right historical conditions. Specifically, we argue that the political weakening of the UK coal sector was a necessary precursor to the adoption of strong climate policy and the emergence of a structural consensus towards accelerating climate ambition, as compared with Germany where consolidation of the coal regime has been a major factor in the country's failure to meet its emissions targets. We show how business associations and labour groups in the UK were a key supportive coalition for early climate action, while in Germany industry and organized labour have been key actors obstructing and delaying the passage of pro-climate reforms. Our study raises questions about the prospects for energy transitions that are both just and rapid, particularly by discussing the trade-offs between cost-effectiveness, speed, and distributional concerns.

EDITOR'S NOTES—COUNTERFACTUAL THINKING: A NECESSITY FOR PRACTITIONERS AND RESEARCHERS

EDITOR'S NOTES—COUNTERFACTUAL THINKING: A NECESSITY FOR PRACTITIONERS AND RESEARCHERS

Comparing Children’s Behavior Problems in Biological Married, Biological Cohabitating, and Stepmother Families in the UK

A large body of research shows that children who live with two married biological parents have lower levels of externalizing and internalizing behavior problems compared to their peers in other family structure, including cohabitating biological families. Such patterns suggest that marriage provides a uniquely protective family environment, though we know less about children in the obvious counterfactual case: married stepfamilies. While research suggests children with stepfathers have more behavior problems than those living with married biological parents, we know little about how children with stepmothers fare, or how children with stepparents fare compared to those living with cohabiting biological parents. We use the Millennium Cohort Study (MCS) sweep 6 to compare children living with married biological parents, married fathers and stepmothers, and cohabiting biological parents. We find that family structure has no significant relationship with children's internalizing behavior problems, but that children living with a stepmother and biological cohabitating families exhibit more externalizing behavior problems than do those living with married biological parents. Covariates that indicate both physical and social family environments must be considered together to explain differences in married-parent families on externalizing behavior problems.

Creating the Conditions for Intergenerational Justice: Social Capital and Compliance

Creating the Conditions for Intergenerational Justice: Social Capital and Compliance

Endogenous, Regime-Switching Hedonic Estimation of Commercial Waterway Water Quality Impact on Home Values in the Alabama Black Belt

We are the first researchers to investigate the impact of the water quality of a commercial marine waterway on the housing market. We use housing data for properties in proximity to the major waterways and minor tributaries of the Gulf Intracoastal Waterway in the Alabama Black Belt. Research shows a positive relationship between water quality and the value of waterfront properties. We use waterway impairment as a proxy to measure water quality to conduct a counterfactual analysis based on a unique endogenous regime-switching hedonic price model. We find significant empirical evidence that properties in proximity to the Black Belt’s major waterways could depreciate by approximately 22% compared to the counterfactual case of properties located on or near minor tributaries. A possible implication might be that homeowners prefer properties on or near minor tributaries of the Alabama Black Belt because of perceptions that the water quality is better than on major commercial waterways.

Quantifying the impact of the Tokyo Olympics on COVID-19 cases using synthetic control methods

This paper uses a synthetic control method (SCM) and a Ridge Augmented SCM to estimate the impact of holding the Tokyo Olympic games on the number of newly confirmed COVID-19 cases in Tokyo (Japan). Our analysis with these methods enables us to estimate the causal impact of the Tokyo Olympics on COVID-19 cases by constructing counterfactual COVID-19 cases for Tokyo (Japan) as the optimal weighted average of COVID-19 cases of OECD countries that are not affected by holding the Olympics through a data-driven approach. Based on reliable estimates obtained from different analytical settings, we find that, compared to the counterfactuals, holding the Tokyo Olympics significantly increased the daily average number of COVID-19 cases by 105 to 132 cases in Tokyo (47 to 65 cases in Japan as a whole) per million people. This result suggests that holding the Olympics likely led to the spread of COVID-19 infection in Tokyo (Japan).

Defining Explanation and Explanatory Depth in XAI

Explainable artificial intelligence (XAI) aims to help people understand black box algorithms, particularly of their outputs. But what are these explanations and when is one explanation better than another? The manipulationist definition of explanation from the philosophy of science offers good answers to these questions, holding that an explanation consists of a generalization that shows what happens in counterfactual cases. Furthermore, when it comes to explanatory depth this account holds that a generalization that has more abstract variables, is broader in scope and/or more accurate is better. By applying these definitions and contrasting them with alternative definitions in the XAI literature I hope to help clarify what a good explanation is for AI.

Estimation of vaccine efficacy for variants that emerge afterthe placebo group is vaccinated.

SARS‐CoV‐2 continues to evolve and the vaccine efficacy against variants is challenging to estimate. It is now common in phase III vaccine trials to provide vaccine to those randomized to placebo once efficacy has been demonstrated, precluding a direct assessment of placebo controlled vaccine efficacy after placebo vaccination. In this work, we extend methods developed for estimating vaccine efficacy post placebo vaccination to allow variant specific time varying vaccine efficacy, where time is measured since vaccination. The key idea is to infer counterfactual strain specific placebo case counts by using surveillance data that provide the proportions of the different strains. This blending of clinical trial and observational data allows estimation of strain‐specific time varying vaccine efficacy, or sieve effects, including for strains that emerge after placebo vaccination. The key requirements are that the surveillance strain distribution accurately reflects the strain distribution for a placebo group throughout follow‐up after placebo group vaccination, and that at least one strain is present before and after placebo vaccination. For illustration, we develop a Poisson approach for an idealized design under a rare disease assumption and then use a proportional hazards model to address staggered entry, staggered crossover, and smoothly varying strain specific vaccine efficacy. We evaluate these methods by theoretical work and simulations, and demonstrate that useful estimation of the efficacy profile is possible for strains that emerge after vaccination of the placebo group. An important principle is to incorporate sensitivity analyses to guard against misspecification of the strain distribution.

The rewards of fiscal consolidations: Sovereign spreads and confidence effects

This paper investigates the effects of fiscal consolidation announcements on sovereign spreads using high frequency data for a panel of 21 emerging market economies during 2000–18. We construct a novel dataset using a global news database to identify the precise announcement date of fiscal consolidation actions. Our results show that sovereign spreads barely decline after fiscal announcements from the executive branch. Nonetheless, spreads decline significantly following news that austerity measures have been approved by the legislature (congress or parliament), particularly in economies with high sovereign spreads, debt levels, and low credit-risk ratings. These effects are even larger in the presence of an IMF program. In addition, we find that consolidation announcements are less contractionary when sovereign spreads decline. The reduction in output is a third of the counterfactual case in which spreads do not respond to announcements. These results show that confidence effects, in the form of lower sovereign spreads, are an important transmission channel of fiscal policy actions. We also find that the role of confidence effects increases with the level of spreads, such that countries with high spread levels stand to benefit the most from putting in place credible austerity packages.

The impact of the COVID-19 pandemic and associated suppression measures on the burden of tuberculosis in India

BackgroundTuberculosis (TB) is a major cause of death globally. India carries the highest share of the global TB burden. The COVID-19 pandemic has severely impacted diagnosis of TB in India, yet there is limited data on how TB case reporting has changed since the pandemic began and which factors determine differences in case notification.MethodsWe utilized publicly available data on TB case reporting through the Indian Central TB Division from January 2017 through April of 2021 (prior to the first COVID-19 related lockdown). Using a Poisson model, we estimated seasonal and yearly patterns in TB case notification in India from January 2017 through February 2020 and extended this estimate as the counterfactual expected TB cases notified from March 2020 through April 2021. We characterized the differences in case notification observed and those expected in the absence of the pandemic by State and Territory. We then performed a linear regression to examine the relationship between the logit ratio of reported TB to counterfactual cases and mask use, mobility, daily hospitalizations/100,000 population, and public/total TB case reporting.ResultsWe found 1,320,203 expected cases of TB (95% uncertainty interval (UI) 1,309,612 to 1,330,693) were not reported during the period from March 2020 through April 2021. This represents a 63.3% difference (95% UI 62.8 to 63.8) in reporting. We found that mobility data and average hospital admissions per month per population were correlated with differences in TB case notification, compared to the counterfactual in the absence of the pandemic (p > 0.001).ConclusionThere was a large difference between reported TB cases in India and those expected in the absence of the pandemic. This information can help inform the Indian TB program as they consider interventions to accelerate case finding and notification once the pandemic related TB service disruptions improve. Mobility data and hospital admissions are surrogate measures that correlate with a greater difference in reported/expected TB cases and may correlate with a disruption in TB diagnostic services. However, further research is needed to clarify this association and identify other key contributors to gaps in TB case notifications in India.

Limitations of econometric evaluation of nonrandomized residential energy efficiency programs: A case study of Northern California rebate programs

Abstract Residential energy efficiency programs play an important role in combating climate change. More precise quantification of the magnitude and timing of energy savings would bring large system benefits, allowing closer integration of energy efficiency into resource adequacy planning and balancing variable renewable electricity. However, it is often difficult to quantify the efficacy of an energy efficiency intervention, because doing so requires consideration of a hypothetical counterfactual case in which there was no intervention, and randomized control trials are often implausible. Although quasi-experimental econometric evaluation sometimes works well, we find that for a set of energy efficiency rebate programs in Northern California, a naïve interpretation of econometric measurement finds that rebate participation is associated with an average increase in electricity consumption of 7.2% [4.5%, 10.1%], varying in magnitude and sign depending on the type of appliance or service covered by the rebate. A subsequent household survey on appliance purchasing behavior and analysis of utility customer outreach data suggest that this regression approach is likely measuring the gross impact of buying a new appliance but fails to adequately capture a counterfactual comparison. Indeed, it is unclear whether it is even possible to construct a suitable counterfactual for econometric analyses of these rebate programs using data generally available to electric utilities. We view these results as an illustration of a limitation of econometric methods of program evaluation and the importance of weighing engineering modeling and other imperfect methods against one another when attempting to provide useful evaluations of real-world policy interventions.

Estimating the Impact of Low Influenza Activity in 2020 on Population Immunity and Future Influenza Seasons in the United States.

BackgroundInfluenza activity in the 2020–2021 season was remarkably low, likely due to implementation of public health preventive measures such as social distancing, mask wearing, and school closure. With waning immunity, the impact of low influenza activity in the 2020–2021 season on the following season is unknown.MethodsWe built a multistrain compartmental model that captures immunity over multiple influenza seasons in the United States. Compared with the counterfactual case, where influenza activity remained at the normal level in 2020–2021, we estimated the change in the number of hospitalizations when the transmission rate was decreased by 20% in 2020–2021. We varied the level of vaccine uptake and effectiveness in 2021–2022. We measured the change in population immunity over time by varying the number of seasons with lowered influenza activity.ResultsWith the lowered influenza activity in 2020–2021, the model estimated 102 000 (95% CI, 57 000–152 000) additional hospitalizations in 2021–2022, without changes in vaccine uptake and effectiveness. The estimated changes in hospitalizations varied depending on the level of vaccine uptake and effectiveness in the following year. Achieving a 50% increase in vaccine coverage was necessary to avert the expected increase in hospitalization in the next influenza season. If the low influenza activity were to continue over several seasons, population immunity would remain low during those seasons, with 48% of the population susceptible to influenza infection.ConclusionsOur study projected a large compensatory influenza season in 2021–2022 due to a light season in 2020–2021. However, higher influenza vaccine uptake would reduce this projected increase in influenza.

Biohydrogen: A life cycle assessment and comparison with alternative low-carbon production routes in UK

This study focuses on the production of hydrogen from municipal solid waste (MSW) for applications in transportation. A life cycle assessment (LCA) was conducted on a semi-commercial advanced gasification process for Biohydrogen (Bio-H2) production from MSW to evaluate its environmental impact on five impact categories: Climate Change, Acidification, Eutrophication Fresh Water, Ecotoxicity Freshwater and Photochemical Ozone Formation (human health). The biogenic composition of waste and the effect of carbon sequestration were analysed for Bio-H2, uncovering a net-negative carbon process. The counterfactual case of MSW incineration further bolsters the carbon savings associated to Bio-H2. The production of Bio-H2 from waste is proven to be competitive against alternative hydrogen productions routes, namely blue hydrogen (Blue-H2) produced via steam methane reforming/autothermal reforming coupled with carbon capture and storage (CCS), and green hydrogen (Green-H2) from solar and offshore wind, with respect to climate change. These climate change advantages are shown to carry forward in the context of decarbonisation of electricity grid mix, as analysed by scenarios taken for 2030 and ‘net-zero’ 2050.

No paradox here? Improving theory and testing of the nuclear stability–instability paradox with synthetic counterfactuals

This article contributes to both the theoretical elaboration and empirical testing of the ‘stability–instability paradox’, the proposition that while nuclear weapons deter nuclear war, they also increase conventional conflict among nuclear-armed states. Some recent research has found support for the paradox, but quantitative studies tend to pool all international dyads while qualitative and theoretical studies focus almost exclusively on the USA–USSR and India–Pakistan dyads. This article argues that existing empirical tests lack clearly relevant counterfactual cases, and are vulnerable to a number of inferential problems, including selection on the dependent variable, unintentionally biased inference, and extrapolation from irrelevant cases. The limited evidentiary base coincides with a lack of consideration of the theoretical conditions under which the paradox might apply. To address these issues this article theorizes some scope conditions for the paradox. It then applies synthetic control, a quantitative method for valid comparison when appropriate counterfactual cases are lacking, to model international conflict between India–Pakistan, China–India, and North Korea–USA, before and after nuclearization. The article finds only limited support for the paradox when considered as a general theory, or within the theorized scope conditions based on the balance of resolve and power within each dyad.

Introduction: Causation, inferences, and solution types in configurational comparative methods

This special issue addresses questions of causality and validity of different solution types in configurational comparative methods (CCMs). First, what main parameters characterize the debate about correct causal interpretation of solution types? Second, to what extent has this debate been linked to a theory of causation? The special issue contribution by Mahoney and Acosta bases qualitative comparative analysis (QCA) within a regularity theory of causation integrating type-level inferences and counterfactual cases. Swiatczak clarifies how the different algorithms underlying QCA and Coincidence Analysis (CNA) produce non-identical models. Baumgartner defines and benchmarks QCA solution types against the search target of minimal robust sufficiency. Alamos-Concha et al. identify the conservative solution as most appropriate for a multimethod design combining a counterfactual causal understanding at the cross-case level with an in-depth mechanistic explanation at the within-case level. Finally, Mahoney and Owen develop a general set-theoretic framework for the study of necessity and sufficiency in quantitative research using a counterfactual understanding of causality. Our introduction reviews the state of the art, identifies current limitations and open questions regarding the theoretical basis for causal interpretation of QCA solutions.