Disciplines, institutions and organizations: impact assessments in context

Abstract

This paper locates impact assessment research within the disciplinary constructs of agricultural economics, the institutions (rules of the game) and organizations (structures governed by these rules/norms) of agricultural innovation. The paper analyses the disciplinary limitations in impact assessment research. In the hard science culture of agricultural research the social sciences, closer to the reality of complex technological contexts, are less prestigious. Agricultural economics, the disciplinary root of impact assessment, imposes a major constraint by making institutional contexts and variations intractable and analysed at best as disturbances. Sophisticated econometric methodologies substitute for information on institutions, systems components and relationships. The paper presents an analytical framework to distinguish between the institutional and organizational variables in the agricultural innovation system. The case study of institutional and organizational change in Indian agricultural research and extension is used to demonstrate the role of institutions in determining the impact of agricultural science and technology. Impact assessment research is institutionally conditioned to look for end point consequences of research, such as yield increase, cost reduction, or farm income. This body of knowledge ignores crucial requirements such as the institutional framework (hierarchy for instance) and actors that facilitate these end point impacts. Mainstream impact assessment literature is not concerned about the important relationships among these institutions and actors, or about the impact of knowledge on these institutions and organizations. In the quest for end point consequences, estimated often as rates of return to investment, impact assessment has overlooked important requirements and relationships within agricultural innovation systems. An analysis of institutional learning using the key elements of an evaluation culture as proposed by program evaluators is used to demonstrate the need for impact assessment to engage with institutional questions and build the capacity for institutional learning. There is a felt need to move from impact assessment to a larger evaluation culture. Agricultural research evaluation has much to learn from regular program evaluation. This calls for more social science research besides neoclassical agricultural economics, and active interaction among the natural and social scientists, extensionists, farming communities, policy makers and other actors. Effective evaluation in general and impact assessment in particular depends on the institutional learning capacity of the actors, especially scientists and policy makers.