Do LED lightbulbs save natural gas? Interpreting simultaneous cross-energy program impacts using electricity and natural gas billing data

Abstract

Local and state governments have made significant advances in creating, implementing, and evaluating energy efficiency programs in the buildings sector. Evaluations commonly use ex-post statistical models to complement ex-ante engineering estimates when determining program impacts. A critical assumption of data-driven evaluations is that reductions would not occur in the absence of the program. This assumption is difficult to test, particularly if other unobserved changes to the building’s energy profile coincide with the program’s adoption. We provide a method to detect a class of unobserved simultaneous cross-energy changes in a building’s energy profile by examining the treatment effects of electricity-only programs on natural gas use and vice versa. We apply the method to a panel of residential energy efficiency implementations with monthly electricity and gas data from 2010 to 2016 in the City of Palo Alto, California. Using difference-in-differences and event history analyses, we find evidence of significant gas reductions estimated for some electricity-only programs, suggesting that households implemented unobserved changes at the same time as those programs. Our results highlight how data-driven analyses may not adequately estimate program impacts, and the value of simultaneous electricity and natural gas measurements for detecting and interpreting unobserved changes to energy use at the household level. Lastly, we present evidence that energy savings from non-monetary interventions can exceed those which offer financial rewards for energy efficiency.