Formative, embedded evaluation to strengthen interdisciplinary team science: Results of a 4-year, mixed methods, multi-country case study

Abstract

Evaluation of interdisciplinary, team science research initiatives is an evolving and challenging field. This descriptive, longitudinal, mixed methods case study examined how an embedded, formative evaluation approach contributed to team science in the interdisciplinary Research into Policy to Enhance Physical Activity (REPOPA) project, which focused on physical activity policymaking in six European countries with divergent policy systems and researcher–policymaker networks. We assessed internal project collaboration, communication, and networking in four annual data collection cycles with REPOPA team members. Data were collected using work package team and individual interviews, and quantitative collaboration and social network questionnaires. Interviews were content analyzed; social networks among team members and with external stakeholder were examined; collaboration scores were compared across 4 years using analysis of variance (ANOVA). Annual monitoring reports with action recommendations were prepared and discussed with consortium members. Results revealed consistently high response rates. Collaboration and communication scores, high at baseline, improved slightly, but ANOVA results were nonsignificant. Internal network changes tracked closely with implementation progress. External stakeholders were primarily governmental, with a marked shift from local/provincial level to national/international during the project. Diversity (disciplinary, organizational, and geopolitical) was a project asset influencing and also challenging collaboration, implementation, and knowledge translation strategies. In conclusion, formative evaluation using an embedded, participatory approach demonstrated utility, acceptability, and researcher engagement. A trusting relationship between evaluators and other project members built on joint identification of team science objectives for the evaluation at project outset, codeveloping guiding principles, and encouraging team reflexivity throughout the evaluation.