Evaluation Of Public Health Interventions Research Articles

HPR62 Economic Evaluations of Public Health Interventions on COVID-19: A Systematic Review

Due to the rapid spread of COVID-19, governments worldwide have implemented drastic policy measures to contain the transmission of the virus, of which have caused economic downturn and social calamity. There is paucity of data on the full economic cost (EEs) and cost-effectiveness of different policy measures currently used in containing the spread of COVID-19. Objective This systematic review was conducted to summarize the existing evidence on the cost and cost-effectiveness of a range of policy measures used to contain COVID-19 transmission.

Intimate partner violence and mental health: lessons from the COVID-19 pandemic.

Intimate partner violence and mental health: lessons from the COVID-19 pandemic.

A model for effective partnership working to support programme evaluation

The use of multi-agency partnerships, including research-practice partnerships, to facilitate the development, implementation and evaluation of public health interventions has expanded in recent years. However, gaps remain in the understanding of influences on partnership working, and their capacity to facilitate and use evaluation, as well as the characteristics which lead to partnership effectiveness. We applied qualitative methods to explore experiences of stakeholders who were involved in partnerships to deliver and evaluate a national physical activity programme. We combined thematic and network analysis, and drew on concepts of evaluation use, knowledge exchange and organisational systems to interpret our findings and develop a conceptual model of the relationships between partnership characteristics and processes. Our model identifies key partnership characteristics such as high levels of engagement, regular communication and continuity. Furthermore, it highlights the importance of implementing organisational structures and systems to support effective partnership working, knowledge exchange and capacity building.

The case for using mixed methods for designing, implementing, and disseminating evidence-based interventions for public health practice.

A well-recognized gap exists between findings from public health research and their use in public health practice to improve outcomes. The World Health Organization (WHO) has identified implementation research as vital to improving the adaptation and dissemination of public health interventions into real-world settings. Implementation research encompasses multimethod evaluation approaches; 'mixed methods', the planned integration of qualitative and quantitative methods, is a key tool. We argue that mixed methods designs are crucial for design and evaluation of public health interventions, provide illustrative case studies, discuss key analytic approaches in mixed methods design, identify resources for mixed methods research and advocate for more training. On behalf of the International Network for Epidemiology in Policy (INEP), an international non-profit organization of 24 member societies that promotes the development of equitable, evidence-based health policies, we call for increased collaboration between qualitative and quantitative research teams to improve the design and evaluation of public health interventions.

Understanding social pathology of disease causation and socio-cultural factors of corona virus (COVID-19) in South-West, Nigeria

The new SARS-CoV-2 coronavirus disease (COVID-19) started in Wuhan City of China on December 31st 2019As at August 3,2020 a total of 18,056,310 million cases had been diagnosed globally with over 689,219 deaths with cases in Nigeria snowballing gradually becoming lethal. Given Nigeria’s socio-economic and demographic significance to African continent, it is imperative to understand the cultural norms that may aid or obstructs prevention and treatment of the disease in order to halt its transmission. Data for study came from the Nigeria Centre for Disease Control and other publicly available data sources supported with PEN-3 cultural model developed in 1989 by Airhihenbuwa. The model places culture at the core of the development, implementation and evaluation of successful public health interventions. COVID-19 transmission increases with large population concentration in urban areas and proximity to major entry points to other adjacent states and countries. The paper suggested that dominant cultures, civilization and religious practices should be adhered to, adopted as the case may be for restrictions such physical distancing, hand hygiene, use of face masks and another prophylactic regimen to flatten the curve of the pandemic in Nigeria and likely occurrence of similar disease in future.

Measuring and valuing broader impacts in public health: Development of a sanitation-related quality of life instrument in Maputo, Mozambique.

Two billion people globally lack access to a basic toilet. While improving sanitation reduces infectious disease, toilet users often identify privacy, safety and dignity as more important. However, these outcomes have not been incorporated in sanitation-related economic evaluations. This illustrates the general challenge of outcome measurement and valuation in the economic evaluation of public health interventions, and risks misallocating the US$ 20 billion invested in sanitation in low- and middle-income countries every year. In this study in urban Mozambique, we develop an instrument to measure sanitation-related quality of life (SanQoL). Applying methods from health economics and the capability approach, we develop a descriptive system to measure five attributes identified in prior qualitative research: disgust, health, shame, safety and privacy. Sampling individuals from the intervention and control groups of a sanitation intervention trial, we elicit attribute ranks to value a SanQoL index and assess its validity and reliability. In combination with a measure of time using a sanitation service, SanQoL can quantify incremental benefits in a sanitation-focused cost-effectiveness analysis. After monetary valuation based on willingness to pay, QoL benefits could be summed with health gains in cost-benefit analysis, the most common method in sanitation economic evaluations.

Health-Related Quality of Life Measurement in Public Health.

Patient-reported outcomes are recognized as essential for the evaluation of medical and public health interventions. Over the last 50 years, health-related quality of life (HRQoL) research has grown exponentially from 0 to more than 17,000 papers published annually. We provide an overview of generic HRQoL measures used widely in epidemiological studies, health services research, population studies, and randomized clinical trials [e.g., Medical Outcomes Study SF-36 and the Patient-Reported Outcomes Measurement Information System (PROMIS®)-29]. In addition, we review methods used for economic analysis and calculation of the quality-adjusted life year (QALY). These include the EQ-5D, the Health Utilities Index (HUI), the self-administered Quality of Well-being Scale (QWB-SA), and the Health and Activities Limitation Index (HALex). Furthermore, we consider hybrid measures such as the SF-6D and the PROMIS-Preference (PROPr). The plethora of HRQoL measures has impeded cumulative science because incomparable measures have been used in different studies. Linking among different measures and consensus on standard HRQoL measurement should now be prioritized. In addition, enabling widespread access to common measures is necessary to accelerate future progress.

Swdpwr: A SAS macro and an R package for power calculations in stepped wedge cluster randomized trials

Background and objective:The stepped wedge cluster randomized trial is a study design increasingly used in a wide variety of settings, including public health intervention evaluations, clinical and health service research. Previous studies presenting power calculation methods for stepped wedge designs have focused on continuous outcomes and relied on normal approximations for binary outcomes. These approximations for binary outcomes may or may not be accurate, depending on whether or not the normal approximation to the binomial distribution is reasonable. Although not always accurate, such approximation methods have been widely used for binary outcomes. To improve the approximations for binary outcomes, two new methods for stepped wedge designs (SWDs) of binary outcomes have recently been published. However, these new methods have not been implemented in publicly available software. The objective of this paper is to present power calculation software for SWDs in various settings for both continuous and binary outcomes. Methods: We have developed a SAS macro %swdpwr, an R package swdpwr and a Shiny app for power calculations in SWDs. Different scenarios including cross-sectional and cohort designs, binary and continuous outcomes, marginal and conditional models, three link functions, with and without time effects under exchangeable, nested exchangeable and block exchangeable correlation structures are accommodated in this software. Unequal numbers of clusters per sequence are also allowed. Power calculations for a closed cohort employ a block exchangeable within-cluster correlation structure that accounts for three intracluster (intraclass) correlations: the within-period, between-period, and within-individual correlations. Cross-sectional cohorts allow for nested exchangeable or exchangeable correlation structures defined by the within-period and the between-period intracluster correlations only. Our software assumes a complete design and equal cluster-period sizes. While the methods accommodate correlation structures of constant within-period intracluster correlation coefficient (ICC) as well as a different within- and between-period ICC, it does not allow the between-period ICC to decay. Results:swdpwr provides an efficient tool to support investigators in the design and analysis of stepped wedge cluster randomized trials. swdpwr addresses the implementation gap between newly proposed methodology and their application to obtain more accurate power calculations in SWDs. Conclusions: In an effort to make computationally efficient (and non-simulation-based) power methods under both the cross-sectional and closed-cohort designs for continuous and binary outcomes more accessible, we have developed this user-friendly software. swdpwr is implemented under two platforms: SAS and R, satisfying the needs of investigators from various backgrounds. Additionally, the Shiny app enables users who are not able to use SAS or R to implement these methods online straightforwardly.

Using path analysis to test theory of change: a quantitative process evaluation of the MapSan trial

BackgroundAlthough theory-driven evaluations should have empirical components, few evaluations of public health interventions quantitatively test the causal model made explicit in the theory of change (ToC). In the context of a shared sanitation trial (MapSan) in Maputo, Mozambique, we report findings of a quantitative process evaluation assessing intervention implementation, participant response and impacts on hypothesised intermediary outcomes on the pathway to trial health outcomes. We examine the utility of path analysis in testing intervention theory using process indicators from the intervention’s ToC.MethodsProcess data were collected through a cross-sectional survey of intervention and control compounds of the MapSan trial > 24-months post-intervention, sampling adult residents and compound leaders. Indicators of implementation fidelity (dose received, reach) and participant response (participant behaviours, intermediary outcomes) were compared between trial arms. The intervention’s ToC (formalised post-intervention) was converted to an initial structural model with multiple alternative pathways. Path analysis was conducted through linear structural equation modelling (SEM) and generalised SEM (probit model), using a model trimming process and grouped analysis to identify parsimonious models that explained variation in outcomes, incorporating demographics of respondents and compounds.ResultsAmong study compounds, the MapSan intervention was implemented with high fidelity, with a strong participant response in intervention compounds: improvements were made to intermediary outcomes related to sanitation ‘quality’ – latrine cleanliness, maintenance and privacy – but not to handwashing (presence of soap / soap residue). These outcomes varied by intervention type: single-cabin latrines or multiple-cabin blocks (designed for > 20 users). Path analysis suggested that changes in intermediary outcomes were likely driven by direct effects of intervention facilities, with little contribution from hygiene promotion activities nor core elements expected to mediate change: a compound sanitation committee and maintenance fund. A distinct structural model for two compound size subgroups (≤ 20 members vs. > 20 members) explained differences by intervention type, and other contextual factors influenced specific model parameters.ConclusionsWhile process evaluation found that the MapSan intervention achieved sufficient fidelity and participant response, the path analysis approach applied to test the ToC added to understanding of possible ‘mechanisms of change’, and has value in disentangling complex intervention pathways.Trial registrationMapSan trial registration: NCT02362932 Feb-13-2015.

Including the patient's oral health perspective in evidence-based decision-making.

How to approach the assessment of patient-perceived oral health is of fundamental importance for the evaluation of clinical and public health interventions because the patient's assessment should be used as an adjunct to objective dental findings in order to decide which interventions work. This review article aims to provide an overview of the principles, current status, and future outlook for how a patient's oral health perception can and should be assessed. The hierarchical position of dental patient-reported outcomes, oral health-related quality of life (OHRQoL), and dental patient-reported outcome measures within the hierarchical concepts of quality of life and its component, health-related quality of life, is presented. The Mapping Oral Disease Impact with aCommon Metric project is outlined as an international effort to describe current approaches to standardize the measurement of oral impact using the four OHRQoL dimensions of oral function, orofacial pain, orofacial appearance, and psychosocial impact. Ultimately, these four dimensions of OHRQoL provide apractical and psychometrically solid way to collect and analyze OHRQoL data for all oral diseases in all settings, and eventually for all treatments through the use of astandardized, universal measurement tool. This universal impact metric capturing the patient's oral health perspective is the key to moving evidence-based dentistry and value-based oral health care forward.

'What You See is All There is': The Importance of Heuristics in Cost-Benefit Analysis (CBA) and Social Return on Investment (SROI) in the Evaluation of Public Health Interventions.

Health economists are currently debating, with some suspicion, the relative merits of cost-benefit analysis (CBA), grounded in theoretical welfare economics, and the proliferation of social return on investment (SROI), a pragmatic approach of developing a triple-bottom line (social, environmental and financial), but not grounded in welfare theory. We argue, in rather existential terms, that there is a need to understand the role of heuristics, or prior beliefs, in current ‘best practice’ in CBA and SROI. A taxonomy of CBA and SROI is presented, which summarises the origins of the methods, reporting guidance, publication checklist of quality of reporting, who is wanting these analytical approaches, and policy decision rule present. We argue that a bottom-up SROI is best thought of as localised CBA, building stakeholder involvement right into the framing of SROI, perhaps addressing or mitigating the effects of prior heuristics in top-down CBA. Behavioural CBA and social CBA recognise that people are not rational and that sources of value other than willingness to pay may best reflect social values. Standardisation of SROI and comparison with CBA may illuminate the role of prior heuristics and seek to better reflect social value in weighing up the costs and benefits of public health interventions at both a local and societal level.

Prediction Models for Public Health Containment Measures on COVID-19 Using Artificial Intelligence and Machine Learning: A Systematic Review.

Artificial Intelligence (AI) and Machine Learning (ML) have expanded their utilization in different fields of medicine. During the SARS-CoV-2 outbreak, AI and ML were also applied for the evaluation and/or implementation of public health interventions aimed to flatten the epidemiological curve. This systematic review aims to evaluate the effectiveness of the use of AI and ML when applied to public health interventions to contain the spread of SARS-CoV-2. Our findings showed that quarantine should be the best strategy for containing COVID-19. Nationwide lockdown also showed positive impact, whereas social distancing should be considered to be effective only in combination with other interventions including the closure of schools and commercial activities and the limitation of public transportation. Our findings also showed that all the interventions should be initiated early in the pandemic and continued for a sustained period. Despite the study limitation, we concluded that AI and ML could be of help for policy makers to define the strategies for containing the COVID-19 pandemic.

Cost-effectiveness analysis of public health interventions with impacts on health and criminal justice: An applied cross-sectoral analysis of an alcohol misuse intervention.

Cost-effectiveness analyses of health care programs often focus on maximizing health and ignore nonhealth impacts. Assessing the cost-effectiveness of public health interventions from a narrow health care perspective would likely underestimate their full impact, and potentially lead to inefficient decisions about funding. The aim of this study is to provide a practical application of a recently proposed framework for the economic evaluation of public health interventions, evaluating an intervention to reduce alcohol misuse in criminal offenders. This cross-sectoral analysis distinguishes benefits and opportunity costs for different sectors, makes explicit the value judgments required to consider alternative perspectives, and can inform heterogeneous decision makers with different objectives in a transparent manner. Three interventions of increasing intensity are compared: client information leaflet, brief advice, and brief lifestyle counseling. Health outcomes are measured in quality-adjusted life-years and criminal justice outcomes in reconvictions. Costs considered include intervention costs, costs to the NHS and costs to the criminal justice system. The results are presented for four different perspectives: "narrow" health care perspective; criminal justice system perspective; "full" health care perspective; and joint "full" health and criminal justice perspective. Conclusions and recommendations differ according to the normative judgment on the appropriate perspective for the evaluation.

Evidencing the contribution of social enterprise to health and social care: approaches and considerations

PurposeAs the provision of public services in many advanced welfare states has increasingly come to be marked by competition, social enterprises have actively been encouraged by governments to become involved in the delivery of public services. While the evaluation of complex public health interventions has arguably become increasingly more sophisticated, this has not been the case where social enterprise is concerned: evaluation of the actual impacts of social enterprises remains significantly underdeveloped by comparison. This study aims to support the establishment of a robust evidence base for the use of social enterprise as a policy instrument.Design/methodology/approachThis paper assesses the potential of three methodological approaches common in the evaluation of complex public health interventions and applies them to the complex realm of community-led social enterprise.FindingsOnly through the involvement of different comparator groups, based on the research questions addressed, would it be possible to disentangle the embedded characteristics of organisations such as social enterprises. Each of the methods adopted in this research is time-consuming and resource-intensive and requires the researcher to possess advanced skills. Public officials should recognise the complexity and resource-intensive nature of such evaluation and resource it accordingly. If the aim of policymakers is to understand the added value of social enterprise organisations, an integrative research approach combining different research methods and design should be implemented to improve generalisability.Originality/valueThis study applies a range of favoured approaches to evaluate “complex” public health interventions include systematic reviews, realist evaluation and quasi-experimental investigation. However, such evaluation approaches have rarely been applied before in the context of social enterprise.

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines

BackgroundThe complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate ‘lumping’ of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions.MethodsThe first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions.ResultsThe proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective.ConclusionsDespite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.

Evaluation of public health interventions from a complex systems perspective: A research methods review.

Applying a complex systems perspective to public health evaluation may increase the relevance and strength of evidence to improve health and reduce health inequalities. In this review of methods, we aimed to: (i) classify and describe different complex systems methods in evaluation applied to public health; and (ii) examine the kinds of evaluative evidence generated by these different methods. We adapted critical review methods to identify evaluations of public health interventions that used systems methods. We conducted expert consultation, searched electronic databases (Scopus, MEDLINE, Web of Science), and followed citations of relevant systematic reviews. Evaluations were included if they self-identified as using systems- or complexity-informed methods and if they evaluated existing or hypothetical public health interventions. Case studies were selected to illustrate different types of complex systems evaluation. Seventy-four unique studies met our inclusion criteria. A framework was developed to map the included studies onto different stages of the evaluation process, which parallels the planning, delivery, assessment, and further delivery phases of the interventions they seek to inform; these stages include: 1) theorising; 2) prediction (simulation); 3) process evaluation; 4) impact evaluation; and 5) further prediction (simulation). Within this framework, we broadly categorised methodological approaches as mapping, modelling, network analysis and 'system framing' (the application of a complex systems perspective to a range of study designs). Studies frequently applied more than one type of systems method. A range of complex systems methods can be utilised, adapted, or combined to produce different types of evaluative evidence. Further methodological innovation in systems evaluation may generate stronger evidence to improve health and reduce health inequalities in our complex world.

A continuous data driven translational model to evaluate effectiveness of population-level health interventions: case study, smoking ban in public places on hospital admissions for acute coronary events

BackgroundAn important task in developing accurate public health intervention evaluation methods based on historical interrupted time series (ITS) records is to determine the exact lag time between pre- and post-intervention. We propose a novel continuous transitional data-driven hybrid methodology using a non-linear approach based on a combination of stochastic and artificial intelligence methods that facilitate the evaluation of ITS data without knowledge of lag time. Understanding the influence of implemented intervention on outcome(s) is imperative for decision makers in order to manage health systems accurately and in a timely manner.MethodsTo validate a developed hybrid model, we used, as an example, a published dataset based on a real health problem on the effects of the Italian smoking ban in public spaces on hospital admissions for acute coronary events. We employed a continuous methodology based on data preprocessing to identify linear and nonlinear components in which autoregressive moving average and generalized structure group method of data handling were combined to model stochastic and nonlinear components of ITS. We analyzed the rate of admission for acute coronary events from January 2002 to November 2006 using this new data-driven hybrid methodology that allowed for long-term outcome prediction.ResultsOur results showed the Pearson correlation coefficient of the proposed combined transitional data-driven model exhibited an average of 17.74% enhancement from the single stochastic model and 2.05% from the nonlinear model. In addition, data demonstrated that the developed model improved the mean absolute percentage error and correlation coefficient values for which 2.77% and 0.89 were found compared to 4.02% and 0.76, respectively. Importantly, this model does not use any predefined lag time between pre- and post-intervention.ConclusionsMost of the previous studies employed the linear regression and considered a lag time to interpret the impact of intervention on public health outcome. The proposed hybrid methodology improved ITS prediction from conventional methods and could be used as a reliable alternative in public health intervention evaluation.

Reflection on modern methods: a common error in the segmented regression parameterization of interrupted time-series analyses

Interrupted time-series (ITS) designs are a robust and increasingly popular non-randomized study design for strong causal inference in the evaluation of public health interventions. One of the most common techniques for model parameterization in the analysis of ITS designs is segmented regression, which uses a series of indicators and linear terms to represent the level and trend of the time-series before and after an intervention. In this article, we highlight an important error often presented in tutorials and published peer-reviewed papers using segmented regression parameterization for the analyses of ITS designs. We show that researchers cannot simply use the product between their calendar time variable and the indicator variable indicating pre- versus post-intervention time periods to represent the post-intervention linear segment. If researchers use this often-presented parameterization, they will get an erroneous result for the level change in their time-series. We show that researchers must take care to use the product between their intervention variable and the time elapsed since the start of the intervention, rather than the time since the beginning of their study. Thus, the second linear segment of the time-series indexing the post-intervention level and trend should be zero before intervention implementation and begin by counting from zero, rather than counting from the time elapsed since the beginning of the study. We hope that this article can clarify segmented regression parameterization for the analysis of ITS designs and help researchers avoid confusing and erroneous results in the level changes of their time-series.

Can synthetic controls improve causal inference in interrupted time series evaluations of public health interventions?

Interrupted time series designs are a valuable quasi-experimental approach for evaluating public health interventions. Interrupted time series extends a single group pre-post comparison by using multiple time points to control for underlying trends. But history bias-confounding by unexpected events occurring at the same time of the intervention-threatens the validity of this design and limits causal inference. Synthetic control methodology, a popular data-driven technique for deriving a control series from a pool of unexposed populations, is increasingly recommended. In this paper, we evaluate if and when synthetic controls can strengthen an interrupted time series design. First, we summarize the main observational study designs used in evaluative research, highlighting their respective uses, strengths, biases and design extensions for addressing these biases. Second, we outline when the use of synthetic controls can strengthen interrupted time series studies and when their combined use may be problematic. Third, we provide recommendations for using synthetic controls in interrupted time series and, using a real-world example, we illustrate the potential pitfalls of using a data-driven approach to identify a suitable control series. Finally, we emphasize the importance of theoretical approaches for informing study design and argue that synthetic control methods are not always well suited for generating a counterfactual that minimizes critical threats to interrupted time series studies. Advances in synthetic control methods bring new opportunities to conduct rigorous research in evaluating public health interventions. However, incorporating synthetic controls in interrupted time series studies may not always nullify important threats to validity nor improve causal inference.

Evaluation of Public Health Mass Campaign: Hesitancy and Event RatiosMethod

Several public health interventions administer drugs and vaccines to large number of people for preemptive or reactive intervention. These vaccines and drugs were developed in very narrow conditions in clinical trials and we therefore have to continuously monitor the efficacy in the field. This paper proposes a method to evaluate the efficacy in the field using data collected from the intervention coverage survey and surveillance. The design is closely related to cohort prospective studies except that the sample size is not fixed upfront and the measure of association not affected by the population denominator.It uses two new terms: hesitancy ratio denoted β, which is the ratio of the target that does to receive the intervention to the target that received it; event ratio is denoted by α is the ratio of health event or disease being prevented absolute incidence in the intervention group to that is in control group. It further shows that risk ratio (RR) can be calculated from these two parameters by taking their products. i.e. RR = α β. This method is therefore called Hesitancy and Event Ratios (HER) Method.We can conclude that this method is scientifically sound and can be used in the evaluation of public health intervention in the field. We recommend that this method be included in routine monitoring of programs for efficacy evaluation.