Evaluating a complex health promotion intervention: case application of three systems methods

Abstract

Persistent challenges to public health and well-being are being addressed though complex interventions. Complex interventions are intentional processes designed to shift and realign the form and function of targeted system patterns by changing underlying system dynamics, structures, and conditions. The IM40 initiative was a complex health promotion intervention designed to address adolescent health and well-being through a developmental assets approach. The initiative sought to influence several levels simultaneously (e.g. individual, professional, procedural, and policy), compounding the complexity for implementation and evaluation. The purpose of this case study is to present the development and initiation of an evaluation approach based on systems thinking and complexity science principles, concepts, and methods. We used three systems methods, Group Model Building (GMB), viable systems model (VSM) assessment, and Social Network Analysis (SNA) to identify and examine underlying patterns and structures that influenced system-wide behaviors. Each method focused on a general systems-oriented question and illuminated stakeholder views of the IM40 theory of change, adaptive organization of multiple independent institutions, and relational ties among system actors. Collectively, the methods enabled us to describe systems change over time and examine shifts in boundaries, relationships, perspectives, and dynamics. The implications of this approach relative to typical evaluation activities in public health practice are contrasted and discussed.