Evaluating dispute resolution experiments

Abstract

W e need to do a better job of documenting and evaluating the dispute resolution experiments currently underway. All too often, documentation and evaluation are nothing more than an afterthought the focus of attention only when it is too late to record what actually happened. Typically, evaluation consists of less-than-critical reflections by the participants, intervenors, or funders with the most at stake. And, even when documentation and evaluation are handled by independent observers, such efforts tend to be framed according to the observers' interests, and not with reference to overarching questions at the frontier of theory-building in' the dispute resolution field. These are formidable obstacles to improving practice. While most, if not all, of my colleagues in the dispute resolution field would probably agree with these contentions, some of these same people have obstructed serious documentation and evaluation efforts. Indeed, I count myself among the gUilty. First, let me make a distinction between documentation and evaluation. Documentation means recording from beginning to end the expectations, impressions, and reflections of the participants. Evaluation means gauging the success of an effort once it is completed, according to explicit standards or criteria. Documentation should be handled on an ongoing basis, and can often be completed by the participants themselves (although it is desirable to supplement what the participants produce with independent observations by nonpartisan analysts or observers). Evaluation, on the other hand, shoud always be undertaken by trained social scientists who are not involved in the activities under review. I have encountered resistance to independent documentation and evaluation of dispute resolution efforts in at least three real-life instances, and been the source of resistance in one other. It is important to consider the merits of the resisters' arguments, In the first case, the mediator, claiming that communications among the parties were privileged, sought to block all efforts at documentation. He urged the parties to treat any outside request for information or personal views as a breach of