International validation of quality indicators for evaluating priority setting in low income countries: process and key lessons

Lydia Kapiriri

doi:10.1186/s12913-017-2360-7

Abstract

BackgroundWhile there have been efforts to develop frameworks to guide healthcare priority setting; there has been limited focus on evaluation frameworks. Moreover, while the few frameworks identify quality indicators for successful priority setting, they do not provide the users with strategies to verify these indicators. Kapiriri and Martin (Health Care Anal 18:129-147, 2010) developed a framework for evaluating priority setting in low and middle income countries. This framework provides BOTH parameters for successful priority setting and proposes means of their verification. Before its use in real life contexts, this paper presents results from a validation process of the framework.MethodsThe framework validation involved 53 policy makers and priority setting researchers at the global, national and sub-national levels (in Uganda). They were requested to indicate the relative importance of the proposed parameters as well as the feasibility of obtaining the related information. We also pilot tested the proposed means of verification.ResultsAlmost all the respondents evaluated all the parameters, including the contextual factors, as ‘very important’. However, some respondents at the global level thought ‘presence of incentives to comply’, ‘reduced disagreements’, ‘increased public understanding,’ ‘improved institutional accountability’ and ‘meeting the ministry of health objectives’, which could be a reflection of their levels of decision making. All the proposed means of verification were assessed as feasible with the exception of meeting observations which would require an insider. These findings results were consistent with those obtained from the pilot testing.ConclusionsThese findings are relevant to policy makers and researchers involved in priority setting in low and middle income countries. To the best of our knowledge, this is one of the few initiatives that has involved potential users of a framework (at the global and in a Low Income Country) in its validation. The favorable validation of all the parameters at the national and sub-national levels implies that the framework has potential usefulness at those levels, as is. The parameters that were disputed at the global level necessitate further discussion when using the framework at that level. The next step is to use the validated framework in evaluating actual priority setting at the different levels.

Highlights

While there have been efforts to develop frameworks to guide healthcare priority setting; there has been limited focus on evaluation frameworks
While factors such as stakeholder engagement, accountability, the use of evidence, and the role of contextual factors are well described in the literature, the degree to which relevant stakeholders perceive them as relevant when evaluating priority setting processes has not been established. [8,9,10,11,12,13,14,15,16] This paper fills this gap in the literature by presenting findings from a study which validated a framework developed to evaluate priority setting in low income countries
The framework was informed by the evaluation literature, as well as Daniel’s ‘Accountability for Reasonableness’ framework [21], the work of Gibson and colleagues [18] and Sibbald and colleagues [17, 18] coupled with interviews with experts at the global level, and their experience with priority setting in low income countries

Summary

Introduction

While there have been efforts to develop frameworks to guide healthcare priority setting; there has been limited focus on evaluation frameworks. While there has been progress in developing frameworks to guide priority setting for health interventions in Low income countries (LIC), limited efforts have been devoted to developing corresponding evaluation frameworks and/or quality indicators to enable policy makers to evaluate priority setting While factors such as stakeholder engagement (and the differences in their decision making powers, and their legitimacy), accountability, the use of evidence, and the role of contextual factors are well described in the literature, the degree to which relevant stakeholders perceive them as relevant when evaluating priority setting processes has not been established. The framework recognizes the importance of the broader political, economic and cultural context for the success of any priority setting and identifies relevant contextual factors that should be understood in order to achieve a holistic evaluation on which viable, locally-specific, improvement strategies can be based

Objectives

Methods

Results

Discussion

Conclusion