Evaluating Statebuilding Support: Learning from Experience or Judging from Assumptions?

Abstract

Statebuilding has become a major concern of OECD donor countries in recent years. Several conceptual studies and policy documents devoted to the aim of increasing aid effectiveness in fragile states have been produced within the Development Assistance Committee (DAC), the World Bank and the wider development community. All these documents confirm the urgency of effective statebuilding in countries affected by fragility, conflict, and violence. They also confirm the complexity of statebuilding interventions that aim to foster institutions based on legitimate politics, support the provision of citizen security, contribute to processes and institutions of justice, support foundations for thriving economic activity and help improve state revenue and services. At the same time there is broad agreement in both the academic and the practitioners’ community that statebuilding is first and foremost an endogenous process that takes place within a society, is driven by actors and interests from within this very society and can only to a limited degree be influenced from outside. Yet where the limits of external influence lie, and how statebuilding support is best designed and delivered in order to be effective – these are still underexplored questions. Impact evaluations of statebuilding support, conducted on behalf of those agencies that offer such kinds of assistance, are supposed to provide answers. Do evaluations provide the kind of reliable knowledge about “what works and what doesn’t” that would help aid practitioners to improve their interventions?In a recent study for the Danish development agency “DANIDA” a team of researchers from the German Development Institute, including the authors of this chapter, reviewed about 130 evaluations in the field of international statebuilding support with a view to distil the most relevant findings from many years of development practice (Gravingholt, Leininger & Haldenwang 2012). Yet although we could easily identify common themes and findings in the literature we reviewed, a closer look at the methodologies employed in those studies left us puzzled. In fact, it made us suspect that the observed “high convergence of evaluation results and recommendations” across many studies on diverse countries was not so much proof of their robustness as it reflected “a common methodological weakness” (ibid.: 42). As a consequence, findings that seem to confirm conventional wisdom may well turn out to be artefacts.These observations provide the starting point for this paper. It analyses and discusses the design of major statebuilding evaluations, their methodological rigour and the generalisability of their findings. We aim at assessing the quality of the methods and research techniques used in evaluations of statebuilding support and, by extension, the methodological validity of their findings and conclusions.