Evaluating Success and Failure in Crisis Management

Abstract

Crises and disasters come in many shapes and sizes. They range from global pandemics and global financial crises to tsunamis, hurricanes, volcanic ash clouds, bushfires, terrorist attacks, critical infrastructure failures and food contamination episodes. Threats may be locally isolated such as an explosion at a local fireworks factory, or they may cascade across multiple countries and sectors, such as pandemics. No country is immune from the challenge of managing extraordinary threats, and doing so out of their comfort zone of routine policy making. The crisis management challenge involves managing threats ‘on the ground’, as well as the political fallout and societal fears. Populist and journalistic commentary frequently labels crisis management initiatives as having either succeeded or failed. The realities are much more complex. Evaluators confront numerous methodological challenges. These include the careful consideration of multiple and often competing outcomes, differing perceptions, issues of success for whom, and gray areas stemming from shortfalls and lack of evidence, as well as variations over time. Despite such complexity, some key themes appear continually across evaluations, from internal reviews to royal commissions and accident inquiries. These pertain to the ways in which evaluations can be shaped heavily or lightly by political agendas, the degree to which evaluating organizations are able to open up, the degree to which gray areas and shortfalls are stumbling blocks in producing findings, and the challenge of producing coherent investigative narratives when many storylines are possible. Ultimately, evaluating crisis initiatives is “political” in nature because it seeks to provide authoritative evaluations that reconcile multiple views, from experts and lawyers to victims and their families.