Background and Reflections on the Life Cycle Assessment Harmonization Project

Abstract

Despite the ever-growing body of life cycle assessment (LCA) literature on electricity generation technologies, inconsistent methods and assumptions hamper comparison across studies and pooling of published results. Synthesis of the body of previous research is necessary to generate robust results to assess and compare environmental performance of different energy technologies for the benefit of policy makers, managers, investors, and citizens. With funding from the U.S. Department of Energy, the National Renewable Energy Laboratory1 initiated the LCA Harmonization Project2 in an effort to rigorously leverage the numerous individual studies to develop collective insights. The goals of this project were to understand the range of published results of LCAs of electricity generation technologies, reduce the variability in published results that stem from inconsistent methods and assumptions, and clarify the central tendency of published estimates to make the collective results of LCAs available to decision makers in the near term. The LCA Harmonization Project's initial focus was evaluating life cycle greenhouse gas (GHG) emissions from electricity generation technologies. Six articles from this first phase of the project are presented in a special supplemental issue of the Journal of Industrial Ecology on Meta-Analysis of LCA: coal (Whitaker et al. 2012), concentrating solar power (Burkhardt et al. 2012), crystalline silicon photovoltaics (PVs) (Hsu et al. 2012), thin-film PVs (Kim et al. 2012), nuclear (Warner and Heath 2012), and wind (Dolan and Heath 2012). As a relatively young field of study, LCA has much to learn from more mature fields in terms of approaches to leveraging existing knowledge for higher-order insights. Meta-analysis is now a robust field within the biomedical and social sciences, whose methods inspired those for the LCA Harmonization Project. The state of the science calls for meta-analysis to be preceded by a systematic review. Key features of systematic review (Neely et al. 2010) that were adopted by the LCA Harmonization Project included a comprehensive search of published literature to ensure no bias by, for instance, publication type (journal, report, etc.); multiple, independent reviews of each candidate reference using predefined screening criteria; and the formation of a multidisciplinary review team composed of LCA experts, technology experts, and literature search experts that met regularly to ensure consistent application of the screening criteria. Results of the systematic review portion of the LCA Harmonization Project were published in the Special Report on Renewable Energy Sources and Climate Change Mitigation of the Intergovernmental Panel on Climate Change (http://srren.ipcc-wg3.de/).3 While the screens applied to each technology differed (and are described in detail in each of the above-referenced articles), they each consisted of three general requirements: employed quality and broadly accepted LCA and GHG accounting methods; reported inputs, scenario/technology characteristics, important assumptions, and results in enough detail to trace and trust the results; and evaluated a technology of modern or near-future relevance. To interpret the results of multiple LCAs of a single technology, a deep understanding must be developed of their methods and assumptions. For electricity generation technologies, key factors include system boundary, assumed lifetime of the technology, impact assessment method (e.g., global warming potentials [GWPs] of assessed GHGs), technological performance parameters such as thermal efficiency and capacity factor, and primary energy resource characteristics such as solar resource and fuel heating value. LCAs differ in these attributes often for legitimate reasons, but their inconsistency hampers direct comparison of the results. Therefore the project developed a meta-analytical procedure called “harmonization” that adjusted the previously published estimates to ones based on a more consistent set of methods and assumptions in two main stages.4 First, system harmonization ensured studies used a consistent set of included processes (e.g., system boundary, set of evaluated GHGs) and metrics (e.g., GWPs). Then technical harmonization set certain key performance parameters or primary energy resource characteristics to consistent values chosen to reflect a modern reference system (typically a modern facility operating in the United States). By reducing variability owing to inconsistent methods and assumptions, the resulting harmonized estimates clarify a technology's central tendency of life cycle GHG emissions in ways useful for certain analytical applications and policy and investment decisions. Nevertheless, the parameter values chosen through harmonization may not reflect those desired by all users. Therefore, in each article, methods were provided for adjusting harmonized results to alternative conditions. Finally, in all articles, the results for each step of harmonization are reported both independently and cumulatively to maximize transparency, enabling users to select which results—for example, based on system rather than technological harmonization steps—are most applicable to their needs. Broadly, system harmonization was accomplished by adding or subtracting an element to achieve a common system boundary. Technical harmonization was accomplished by proportional adjustment of the life cycle GHG emissions estimate to the selected value of the performance parameter or primary energy resource characteristic. Each article describes the details of the calculations for each step of harmonization. All but one article performed “light harmonization,” whereby a larger set of estimates were harmonized at a higher level (e.g., consistent system boundary at the level of life cycle stage), as compared to “full harmonization” (see Burkhardt et al. 2012), which employed a more resource-intensive degree of harmonization on fewer estimates. The full harmonization by Burkhardt and colleagues (2012) included applying consistent global warming intensities of materials (mass GHG emitted per unit mass of a material) within the life cycle inventory and developed a consistent system boundary within life cycle stages. One goal of harmonization is to make the estimates of previously published LCAs more consistent, and therefore comparable. Compared to published results, harmonization has been shown to significantly reduce variability in calculated outcomes (i.e., range, interquartile range). In most cases, the median of published estimates is consistent with that of the harmonized results. When analyses or policies require a one- or two-parameter description of life cycle GHG emissions for generic classes of electricity generation technologies (e.g., point estimate and a measure of variability around that estimate), the results achieved through harmonization provide a more precise initial estimate. Refinements can be achieved through application of the customization methods presented in each harmonization article. By its retrospective nature, any meta-analysis is limited in many ways by the studies selected for analysis. While harmonization attempts to update (e.g., by modernizing assumed thermal efficiencies) and reconcile the scope of previously deficient works (e.g., by adding omitted life cycle stages), it cannot make up for a lack of study of certain technologies or issues. When a technology has been studied less frequently, the conclusions that can be drawn from meta-analysis are likewise limited. Similarly, when design variations within a class of technology have not been studied (e.g., different burners or pollution control technologies for coal, or novel PV manufacturing methods), the distributional characteristics of life cycle GHG emissions (e.g., minimum, maximum, median) may not reflect the true distribution for that technology. For many electricity generation technologies, however, these limitations are countered by their repeated study. Additionally, the inclusion of methods for evaluating different assumptions enables researchers to explore cases not specifically examined in previous literature, thereby helping to mitigate the impact of those literature limitations. It is important to note that reporting distributional characteristics does not imply that harmonization is an assessment of likelihood for life cycle GHG emissions or a predictive tool. In addition, while the effectiveness of a given harmonization step at reducing variability is an indicator of the degree of influence on life cycle GHG emissions, harmonization does not include a formal sensitivity analysis. Finally, the success of harmonization at improving the precision of the set of previously published estimates of life cycle GHG emissions does not imply that the results are more accurate. In areas where new science, methods, or context have emerged, all previously published LCAs will lack this aspect, and harmonization may not be able to consider its impact. An important example of this issue, which is relevant for many electricity generation technologies with articles in this special supplemental issue, is that nearly all previously published LCAs are attributional rather than consequential in nature. Market-mediated effects of the deployment of generation technologies, for instance, how variable-output renewables require addition of some amount of dispatchable reserve capacity to maintain system reliability, are not typically quantified. In some ways this is a system boundary issue, but to retrospectively incorporate consequential impacts into attributional LCAs is not a simple matter of addition given the often complex and context-specific interactions among technologies and markets. Thus the answer could change depending on how the question is asked. For many decisions, knowing the more precise estimate of life cycle GHG emissions achieved through harmonization, albeit considering the technology in isolation, is a useful starting point. It is additionally important to understand that the studies passing the screens used in this project do not represent a statistically independent sample. Clustering of published results owing to the use of similar methods could exist along at least one of three dimensions: multiple estimates reported in the same reference, multiple estimates from the same or similar author groups publishing serially, and multiple references citing the same sources of input data. Clustering, if significant enough, could influence the estimates of central tendency, but estimating the degree of bias introduced and adjusting statistics for this bias is challenging. The LCA Harmonization Project's collection and screening of English-language LCAs on electricity generation technologies provides a foundation from which much additional research could be more efficiently conducted. Additional generation technologies and impact categories besides those analyzed in the six studies included in this special supplemental issue could be assessed. Meta-models could be developed and validated in areas of substantial previous research, and relationships between studied attributes explored. Results of harmonization could be aligned with the performance of specific technologies or mixes of facilities (e.g. a grid mix) to estimate life cycle GHG emissions of real systems. Harmonization of global warming intensities of materials, as in the work by Burkhardt and colleagues (2012), could be extended to other noncombustion generation technologies. Perhaps more importantly, issues not previously studied, or not yet studied fully, that have been identified in each of the six harmonization articles contained in the special supplemental issue should be addressed, for instance, consequential effects of variable renewables on the electrical grid; surface mining impacts on release of soil carbon to the atmosphere; truncation error in process-based LCAs; and the impacts of changes going forward in several aspects of materials, such as utilization efficiency in manufacturing, availability of ores, and substitutability of alternatives. Harmonization is a meta-analytical approach that addresses inconsistency in methods and assumptions of previously published life cycle impact estimates. It has been applied in a rigorous manner to estimates of life cycle GHG emissions from many categories of electricity generation technologies in articles that appear in this special supplemental supplemental issue, reducing the variability and clarifying the central tendency of those estimates in ways useful for decision makers and analysts. Each article took a slightly different approach, demonstrating the flexibility of the harmonization approach. Each article also discusses limitations of the current research, and the state of knowledge and of harmonization, pointing toward a path of extending and improving the meta-analysis of LCAs. The LCA Harmonization Project has been generously funded by the Office of Energy Efficiency and Renewable Energy of the U.S. Department of Energy under contract no. DE-AC36-08-GO28308. All coauthors of the six studies included in this special issue are thanked for their contributions, as well as the peer reviewers of their articles and audience members at various presentations of the results of this project. Garvin Heath is a senior scientist and Margaret Mann a senior engineer and group manager in the Strategic Energy Analysis Center of the National Renewable Energy Laboratory, Golden, Colorado, USA.