Health System Impact of the BHOMA Study: A Stepped-Wedge Cluster Randomised Trial

Abstract

Introduction: Health system strengthening has become an important global agenda due to increased recognition that poor health systems result in poor health outcomes. However, strengthening health systems remains a challenging endeavour because of the complexity inherent in health systems. Health system interventions can result in unpredicted or unintended consequences. This paper explores the health system impact of the Better Health Outcomes through Mentoring and Assessments (BHOMA) which was implemented in three rural districts in Zambia over a period of four years. Methods: Five Health system outcomes formed the basis of evaluating the intervention: Health worker skills, training and supervision, health worker motivation, quality of care (Health worker observation and patient exit interviews) in line with the the postulated theory of change. For each outcome, data collected through three cross-sectional facility surveys were combined in a single analysis to estimate the intervention effect. Intention-to-treat analysis was done according to the phase respective clinics were, at the time they provided outcome data. This allowed for comparision of clusters serving as self-controls when assessed prior to the intervention introduction which was staggered over time in stepped wedged design. Findings: Overall there was improvement in three of the four intervention domains during the project period. After adjusting for sex and age, skills score showed increased by an average of 4.8 percentage points; training and supervision scores increased by 3.9%, while quality of care improved by 1.5 percentage points; all the three domain changes were statistically significant (p<0.05). However, overall health worker motivation score showed a non-significant reduction in motivation score of 0.2%. Interpretation: The BHOMA intervention had a positive effect on three out of four major domains of the intervention. It remains to be seen whether these key system improvement changes translate into a measurable effect on the quality of care and whether they are sustained beyond the project life-span. Further qualitative work is needed to understand how motivation can be enhanced without unduly increasing the cost of health care delivery. Funding Statement: The study was funded by Doris Duke Charitable Foundation (DDCF). Declaration of Interests: The authors declare no conflict of Interest. Ethics Approval Statement: The Study was approved by the University of Zambia Bioethics Committee (UNZA BREC).