A novel element of comprehensive program evaluation: Using a domain matrix design

Abstract

AbstractObjectiveThis brief presents a novel element of program evaluation, termed a domain matrix design, and demonstrates the application of the design element with an example from a collaborative evaluation project focused on the Department of the Air Force's Personal Financial Readiness program.BackgroundUtilizing sound scientific principles and employing a replicable approach to program evaluation, the domain matrix design is an adaptable approach to quantitative data collection for program evaluation that minimizes participant survey burden while maximizing available data.MethodThis goal is primarily achieved through the identification of domains of measurable training outcomes that are pertinent to the evaluation and then the strategic assignment of program participants to survey assessment groups that provide sufficient, but limited, overlap of these domains.ResultsThe proposed approach provides sufficient data on all outcome domains, and analyses can be utilized to examine how each domain is linked to other domains as outlined in the program's logic model, while also minimizing the participant burden and streamlining data collection.ConclusionThis brief serves as an example of bridging evaluation to meet internal program stakeholders' needs with opportunities to contribute to the broader research community, specifically the fields of family science and evaluation. That is, the example project demonstrates how evaluation can meet both the internal needs of program stakeholders (e.g., does this program work for us? what modifications are needed to enhance program effectiveness?) and the external needs of the research community (e.g., broad dissemination of findings to share and compare results).