Evaluating natural experiments in ecology: using synthetic controls in assessments of remotely sensed land treatments.

Abstract

Many important ecological phenomena occur on large spatial scales and/or are unplanned and thus do not easily fit within analytical frameworks that rely on randomization, replication, and interspersed a priori controls for statistical comparison. Analyses of such large-scale, natural experiments are common in the health and econometrics literature, where techniques have been developed to derive insight from large, noisy observational data sets. Here, we apply a technique from this literature, synthetic control, to assess landscape change with remote sensing data. The basic data requirements for synthetic control include (1) a discrete set of treated and untreated units, (2) a known date of treatment intervention, and (3) time series response data that include both pre- and post-treatment outcomes for all units. Synthetic control generates a response metric for treated units relative to a no-action alternative based on prior relationships between treated and unexposed groups. Using simulations and a case study involving a large-scale brush-clearing management event, we show how synthetic control can intuitively infer treatment effect sizes from satellite data, even in the presence of confounding noise from climate anomalies, long-term vegetation dynamics, or sensor errors. We find that accuracy depends on the number and quality of potential control units, highlighting the importance of selecting appropriate control populations. Although we consider the synthetic control approach in the context of natural experiments with remote sensing data, we expect the methodology to have wider utility in ecology, particularly for systems with large, complex, and poorly replicated experimental units.