Measuring management practices in primary health care facilities – development and validation of management practices scorecard in Nigeria

Shunsuke Mabuchi,Yusuke Tsugawa,Olakunle Alonge,Sara Bennett

doi:10.1080/16549716.2020.1763078

Abstract

ABSTRACT Background In low- and middle-income countries, there is scarcity of validated and reliable measurement tools for health facility management, and many interventions to improve primary health care (PHC) facilities are designed without adequate evidence base on what management practices are critical. Objective This article developed and validated a scorecard to measure management practices at primary health care facilities under the performance-based financing (PBF) scheme in Nigeria. Methods Relevant management practice domains and indicators for PHC facilities were determined based on literature review and a prior qualitative study conducted in Nigeria. The domains and indicators were tested for face validity via experts review and organized into an interviewer-administered scorecard. A stratified random sampling of PHC facilities in three States in Nigeria was conducted to assess the reliability and construct validity of the scorecard. Inter-rater reliability using inter-class correlation (ICC) (1, k) was assessed with one-way ANOVA. Exploratory factor analysis (EFA) was conducted to assess the construct validity, and an updated factor structure were developed. Results 32 indicators and 6 management practice domains were initially described. Ordinal responses were derived for each indicator. Data on the scorecard were obtained from 111 PHC facilities. The ICC of mean ratings for each team of judges was 0.94. The EFA identified 6 domains (Stakeholder engagement and communication; Community-level activities; Update of plan and target; Performance management; Staff attention to planning, target, and performance; and Drugs and financial management) and reduced the number of indicators to 17. The average communality of selected items was 0.45, and item per factor ratio was 17:6. Conclusions Despite a few areas for further refinement, this paper presents a reliable and valid scorecard for measuring management practices in PHC facilities. The scorecard can be applied for routine supervisory visits to PHC facilities, and can help accumulate knowledge on facility management, how it affects performance, and how it may be strengthened.

Highlights

In low- and middle-income countries, there is scarcity of validated and reliable measurement tools for health facility management, and many interventions to improve primary health care (PHC) facilities are designed without adequate evidence base on what management practices are critical
We selected a tool to form the foundation for the scorecard based on the following criteria: (i) the tool had gone through a validation process that linked management measures to health facility performance; and (ii) the indicators and measurement approach were consistent with findings from a qualitative case study at primary health care centers (PHCCs) in Nigeria [5] that identified community engagement, performance management and staff management as key management practices
We proposed a sample size (N) of 111 PHCCs to explore a scorecard with 32 indicators or questions (p) for a N:p ratio of 3 as described by Arrindell and van der Ende [21] to be adequate for demonstrating the validity and reliability of questionnaires designed for identifying latent constructs with alpha level of 0.05

Summary

Introduction

In low- and middle-income countries, there is scarcity of validated and reliable measurement tools for health facility management, and many interventions to improve primary health care (PHC) facilities are designed without adequate evidence base on what management practices are critical. Much of the research on primary health care (PHC) management practices in low and middleincome countries (LMICs) is qualitative in nature, employing case studies and realist evaluations to understand which types of management practices work in which settings [1–5]. Without a validated and reliable measurement tool, many interventions to improve the management and performance of primary health care facilities in LMICs are designed without an adequate evidence base on what management practices are critical for improving health facility performance. The lack of measurement tool for management practices further limits the assessment of health system strengthening (HSS) interventions in LMICs

Objectives

Methods

Results

Discussion

Conclusion