Identifying Hidden Risks in Multi-Stakeholder, Dynamic System Development Environments

Abstract

In this paper the authors explore the causes and effects of material weaknesses in multi-stakeholder, dynamic system development environments. In traditional system development, cost, schedule and performance measures and metrics are often used to baseline activity and quantify risk. Yet, system development efforts often fail to deliver the desired outcome or have catastrophic in-progress effects regardless of achieving the desired milestones and metrics. While cost, schedule and performance challenges clearly identify risks, the authors assert that the alignment of beliefs between stakeholders about expectations across system development activities add a level of insight that improves the ability of stakeholders to discover unknown risks and manage to successful system outcomes. To test this assertion the authors developed a probabilistic Bayesian Belief Network informed by retrospective, historical case studies. The model provides a mechanism to understand the stakeholders’ beliefs and how these beliefs directly influence decisions and risks. This model was then tested by an analysis of active system development projects to see if it could identify, in a prospective sense, possible material weaknesses It is envisioned that this approach can be used as an initial analysis tool to baseline system development endeavors and provide continuous assessment in dynamic environments.