Credibility of subgroup analyses by socioeconomic status in public health intervention evaluations: An underappreciated problem?

Greig Inglis,Daryll Archibald,Lawrence Doi,Yvonne Laird,Stephen Malden,Louise Marryat,John Mcateer,Jan Pringle,John Frank

doi:10.1016/j.ssmph.2018.09.010

Abstract

There is increasing interest amongst researchers and policy makers in identifying the effect of public health interventions on health inequalities by socioeconomic status (SES). This issue is typically addressed in evaluation studies through subgroup analyses, where researchers test whether the effect of an intervention differs according to the socioeconomic status of participants. The credibility of such analyses is therefore crucial when making judgements about how an intervention is likely to affect health inequalities, although this issue appears to be rarely considered within public health. The aim of this study was therefore to assess the credibility of subgroup analyses in published evaluations of public health interventions. An established set of 10 credibility criteria for subgroup analyses was applied to a purposively sampled set of 21 evaluation studies, the majority of which focussed on healthy eating interventions, which reported differential intervention effects by SES. While the majority of these studies were found to be otherwise of relatively high quality methodologically, only 8 of the 21 studies met at least 6 of the 10 credibility criteria for subgroup analysis. These findings suggest that the credibility of subgroup analyses conducted within evaluations of public health interventions’ impact on health inequalities may be an underappreciated problem.

Highlights

There is a clear social gradient in the vast majority of health outcomes, whereby morbidity and premature mortality are concentrated amongst the most socioeconomically deprived groups in society
We aimed to purposively sample a diverse set of evaluations of public health interventions that reported differential health impacts by any marker of socioeconomic status (SES). [By “purposive,” we mean a sampling strategy which stopped once we had identified a set of methodological issues, related to subgroup analyses in such studies, which seemed not to be augmented by including further studies – i.e. the yield of insights obtained for the reviewing effort expended was clearly reaching a plateau.] we aimed to identify a pool of intervention studies, that had been already quality-appraised in at least one recent structured review, and which claimed to show an impact on health inequalities by SES
Our inclusion criteria were that studies had to: i) be critically appraised as being of “moderate” to “high” quality in a structured review published in the last decade; ii) report on the evaluation of a public health intervention - meaning programmes or policies delivered at a higher level of aggregation than individual patients; iii) describe a public health intervention that was applicable to high-income countries; iv) evaluate the impact of a public health intervention with a credible study design and analysis (not limited to randomised control trials (RCTs), to allow the inclusion of natural experiments and quasi-experimental designs (Craig et al, 2011)); v) report a differential effect of the intervention by SES

Summary

Introduction

There is a clear social gradient in the vast majority of health outcomes, whereby morbidity and premature mortality are concentrated amongst the most socioeconomically deprived groups in society. Reducing health inequalities between the most and least socioeconomically deprived groups in society has been identified as a priority for policymakers in the UK for nearly four decades, little progress has been made in reducing these inequalities to date (Frank & Haw, 2011; Frank & Haw, 2013; Bleich, Jarlenski, Bell, & LaVeist, 2012; Mackenbach, 2011; McCartney, Popham, Katikireddi, Walsh, & Schofield, 2017).

Objectives

Methods

Results

Discussion

Conclusion