The impact of mismeasurement in performance benchmarking: A Monte Carlo comparison of SFA and DEA with different multi-period budgeting strategies

Abstract

Performance-based budgeting has received increasing attention from public and for-profit organizations in an effort to achieve a fair and balanced allocation of funds among their individual producers or operating units for overall system optimization. Although existing frontier estimation models can be used to measure and rank the performance of each producer, few studies have addressed how the mismeasurement by frontier estimation models affects the budget allocation and system performance. There is therefore a need for analysis of the accuracy of performance assessments in performance-based budgeting. This paper reports the results of a Monte Carlo analysis in which measurement errors are introduced and the system throughput in various experimental scenarios is compared. Each scenario assumes a different multi-period budgeting strategy and production frontier estimation model; the frontier estimation models considered are stochastic frontier analysis (SFA) and data envelopment analysis (DEA). The main results are as follows: (1) the selection of a proper budgeting strategy and benchmark model can lead to substantial improvement in the system throughput; (2) a “peanut butter” strategy outperforms a discriminative strategy in the presence of relatively high measurement errors, but a discriminative strategy is preferred for small measurement errors; (3) frontier estimation models outperform models with randomly-generated ranks even in cases with relatively high measurement errors; (4) SFA outperforms DEA for small measurement errors, but DEA becomes increasingly favorable relative to SFA as the measurement errors increase.