Impact of Diffusion and Variability on Vendor Performance Evaluation

Abstract

Abstract : Performance-Based Logistics (PBL) is an initiative that the Department of Defense (DoD) has targeted for aggressive implementation in FY 2006-2009. It is an initiative intended to improve weapon system logistics outcomes, and to reduce weapon system life-cycle costs. Provider evaluation in PBL is intended to center on clearly specified outcome metrics, and mutually agreed upon goals on those metrics -- with the idea that the DoD knows best what it wants in terms of logistics services, but the vendor may know best how to provide those services. PBL can be seen as an extension of the principle of commander's intent, in which leadership presents goals, but subordinates are encouraged to choose methods and processes. Within the field of behavioral decision making, there is substantial literature which shows that decision makers use suboptimal heuristics to value and negotiate agreements such as PBL contracts for services; these are subject to systematic biases in judgment when evaluating performance. In this paper, the authors draw a connection between the intent of PBL on the one hand, and the literature on biases and heuristics on the other. Specifically, they review the literature on PBL and logistics service measurement as it relates to the following: (1) the distinction between process and outcome measures, and (2) the significance of risk. They then review the literature on behavioral decision making and performance evaluation that relates to these two topics, and develop hypotheses regarding the potential impact of process measurement on outcome measurement, and the absence of stated metrics and goals for the variance (risk) of outcomes. They develop specific, testable hypotheses from this review of the literature; report on a test of these hypotheses using questionnaires asking decision makers to evaluate PBL scenarios; and discuss the implications of their findings in practice.