Analyzing Employee Agreement on Maturity Assessment Tools for Organizations

Abstract

The objective of this paper is to investigate the impact that individual raters have on maturity assessments of organizations in the particular context of business process management (BPM). The hypotheses tested relate to the extent of the impact of individuals on maturity score variances and with the enforcing effect of organizational size on the disagreement among employees within organizations. Eight multilevel random‐effects analyses for eight separate maturity dimensions clarify the intra‐class correlation (agreement) within organizations. The analyses are based on a data set with a strictly hierarchical two‐level data structure of employees (1755) nested within organizations (61). Results show that variance within organizations is significantly larger than zero and is even more important than variance between organizations. We conclude that a large individual background effect exists when rating an organization's business process maturity. In addition, we find that the larger the organizations are, the more disagreement within organizations is visible. Copyright © 2012 John Wiley & Sons, Ltd.