Use Of Logic Models Research Articles

Luton, a Marmot town – using a whole-system approach to tackle health equity

Abstract Issue Luton has high deprivation and health inequalities. We have a borough-wide vision that no one should live in poverty by 2040, and have become a health equity town - committed to Marmot policy objectives and recommendations across all we do. Description Luton is 70th most deprived local authority in England, with stark health inequalities. To deliver the 2040 ambition, 5 pillars have been established, including tackling health inequalities. Luton has become the first Marmot Town in the UK, working with the Institute of Health Equity to develop recommendations and actions relating to the Marmot Policy objectives. Cross-sector system partnership has been key -NHS, businesses, housing, voluntary sector, university. Year 1: we mapped the Marmot recommendations to existing workstreams; completed a mixed-methods evaluation to understand system views of the approach and establish our impact measures. We established steering groups: Children and young people; Housing; Employment and skills; Net zero, they continue to embed recommendations using logic models. We have an overarching strategic approach, alongside smaller scale programmes, to create a movement. Including: grant funding; regular newsletter; training; case studies repository and annual celebration event. Results We have established 15 impact measures across Marmot policy areas to review annually. Qualitative evaluation findings built into work plans include: improved governance and accountability; workforce development across the system; use of logic models; Learning from best practice; sharing knowledge and learning. Lessons The wider agenda of Luton 2040 has enabled health inequalities to be embedded into strategic discussions; improved evidence base and clear narrative to support priorities is helpful; clear focused recommendations helps move work forwards; shared understanding of how we will measure long term impact is vital, communicating to the system is key, and long-term ism challenging. Speakers/Panelists Lee Watson East of England Public Health Training Programme, Luton, UK Ramya Ravindrane East of England Public Health Training Programme, Luton, UK Chimeme Egbutah Luton Borough Council, Luton, UK Nasreen Ali University of Bedfordshire, Luton, UK

Centering racial health equity in systematic reviews Paper 4: A systematic review on the use of logic models and frameworks for methodological conduct of evidence synthesis

ObjectiveTo identify evidence syntheses of health interventions addressing racial health equity reporting the use of equity-focused frameworks and logic models. Study Design and SettingThe search strategy included three sources; a search of three bibliographic databases to identify systematic reviews assessing interventions to improve racial health equity, semi-structured interviews with diverse group and a targeted organization website searches (e.g. NIH, USPSTF) to identify relevant logic models and frameworks. The searches were conducted between January 1, 2020, and January 25, 2023. We used a qualitative approach to identify and describe key characteristics of equity-focused logic models and frameworks used in evidence syntheses. ResultsOf the 153 racial health equity-focused evidence syntheses identified, 2 explicitly used logic models to describe the intervention mechanism. We identified seven existing health equity frameworks from semi-structured interviews and electronic search of key websites that were categorized by stated purpose as providing guidance for 1) research, 2) health policy, 3) digital healthcare solutions, and 4) clinical preventive services. Two out of 7 frameworks included guidance on integrating frameworks or logic models in evidence synthesis while the majority provided contextual information on how to define or consider race or racism as a structural determinant of health. ConclusionThere is limited use of logic models and frameworks in evidence syntheses addressing racial health equity. There is a need for more applied frameworks providing guidance for framing, conducting and interpreting findings of evidence syntheses addressing racial health equity.

Participatory logic modeling in a multi-site initiative to advance implementation science

BackgroundLogic models map the short-term and long-term outcomes that are expected to occur with a program, and thus are an essential tool for evaluation. Funding agencies, especially in the United States (US), have encouraged the use of logic models among their grantees. They also use logic models to clarify expectations for their own funding initiatives. It is increasingly recognized that logic models should be developed through a participatory approach which allows input from those who carry out the program being evaluated. While there are many positive examples of participatory logic modeling, funders have generally not engaged grantees in developing the logic model associated with their own initiatives. This article describes an instance where a US funder of a multi-site initiative fully engaged the funded organizations in developing the initiative logic model. The focus of the case study is Implementation Science Centers in Cancer Control (ISC3), a multi-year initiative funded by the National Cancer Institute.MethodsThe reflective case study was collectively constructed by representatives of the seven centers funded under ISC3. Members of the Cross-Center Evaluation (CCE) Work Group jointly articulated the process through which the logic model was developed and refined. Individual Work Group members contributed descriptions of how their respective centers reviewed and used the logic model. Cross-cutting themes and lessons emerged through CCE Work Group meetings and the writing process.ResultsThe initial logic model for ISC3 changed in significant ways as a result of the input of the funded groups. Authentic participation in the development of the logic model led to strong buy-in among the centers, as evidenced by their utilization. The centers shifted both their evaluation design and their programmatic strategy to better accommodate the expectations reflected in the initiative logic model.ConclusionsThe ISC3 case study demonstrates how participatory logic modeling can be mutually beneficial to funders, grantees and evaluators of multi-site initiatives. Funded groups have important insights about what is feasible and what will be required to achieve the initiative’s stated objectives. They can also help identify the contextual factors that either inhibit or facilitate success, which can then be incorporated into both the logic model and the evaluation design. In addition, when grantees co-develop the logic model, they have a better understanding and appreciation of the funder’s expectations and thus are better positioned to meet those expectations.

Citizen science impact pathways for a positive contribution to public participation in science

Positioning citizen science within the broader historical public engagement framework demonstrates how it has the potential to effectively tackle research and innovation issues. Citizen science approaches have their own challenges, which need to be considered in order to achieve this aim and contribute to wider and deeper public engagement. However, programme evaluations, which discuss lessons learned in engaging the public and other stakeholders with science are rare. To address this gap, we present the H2020-funded DITOs project and discuss the use of logic models in citizen science. We share the project’s assumptions, design considerations for deeper engagement and its impact pathways demonstrating how logic models can be utilised in citizen science to monitor programme effectiveness and for their successful implementation. We hope that this work will inspire citizen science practitioners to use similar tools and by doing so, share their experiences and potential barriers. This knowledge is essential for improving the way citizen science is currently practiced and its impacts to both science and society.

Lessons Learned Using Logic Models to Design and Guide DNP Projects.

A final culminating project that is derived from a practice immersion experience is a critical part of a doctor of nursing practice (DNP) program. Many schools use well-designed quality improvement (QI) projects to meet this requirement, but there is no consensus on which QI methods and tools are the most effective for DNP students to demonstrate before graduation. One Mid-Atlantic public university began using QI process models, one of which was a logic model, to guide their DNP projects. This led to subsequent changes in the curriculum, faculty preparation, and practice partner responsibilities. Many benefits and challenges were identified during the logic model implementation that may benefit other schools seeking to increase consistency and rigor. Although the learning process for students and faculty may be challenging, the outcomes included improved project conceptualization, clarity, concision, and feasibility of goal attainment. Use of logic models expands DNP students' skills to develop and implement a successful project and supports clinical scholarship.

Program Evaluation Through the Use of Logic Models

IntroductionIn order to effectively design and evaluate the effectiveness of a new clinical program or intervention, pharmacists must be equipped with the skills and knowledge that are obtained by familiarity and use of a logic model. Currently, most pharmacy school curricula do not include logic model exercises to instill these necessary skills into the knowledge base of doctor of pharmacy students. This report provides understanding of how a logic model can be permanently implemented into pharmacy curricula in order to develop critical thinking skills that will allow students to become more well-rounded in their future practice of pharmacy. MethodsA 23-point questionnaire was developed by the principal investigator, primarily based on feedback from the student reflection papers and areas of interest with the author's prior experience and use of logic models. They were distributed to assess the knowledge, attitudes, and perspectives of students enrolled or previously enrolled in the course. ResultsQuestionnaires were received from 128 students, representing approximately 32% of those provided the opportunity to participate. The majority of students (72.98%) viewed the potential benefits of learning about logic models favorably. Overall, 64.86% of students agreed that the experience gained through constructing their own logic model was an intellectually stimulating activity. ConclusionsThe logic model is an effective tool that can be used to teach pharmacy students how planned program development would contribute to combating various public health issues.

Selecting Review Outcomes for Systematic Reviews of Public Health Interventions.

For systematic reviews to have an impact on public health, they must report outcomes that are important for decision-making. Systematic reviews of public health interventions, however, have a range of potential end users, and identifying and prioritizing the most important and relevant outcomes represents a considerable challenge.In this commentary, we describe potentially useful approaches that systematic review teams can use to identify review outcomes to best inform public health decision-making. Specifically, we discuss the importance of stakeholder engagement, the use of logic models, consideration of core outcome sets, reviews of the literature on end users' needs and preferences, and the use of decision-making frameworks in the selection and prioritization of outcomes included in reviews.The selection of review outcomes is a critical step in the production of public health reviews that are relevant to those who use them. Utilizing the suggested strategies may help the review teams better achieve this.

Developing logic models to inform public health policy outcome evaluation: an example from tobacco control.

The evaluation of large-scale public health policy interventions often relies on observational designs where attributing causality is challenging. Logic models-visual representations of an intervention's anticipated causal pathway-facilitate the analysis of the most relevant outcomes. We aimed to develop a set of logic models that could be widely used in tobacco policy evaluation. We developed an overarching logic model that reflected the broad categories of outcomes that would be expected following the implementation of tobacco control policies. We subsequently reviewed policy documents to identify the outcomes expected to result from the implementation of each policy and conducted a literature review of existing evaluations to identify further outcomes. The models were revised according to feedbacks from a range of stakeholders. The final models represented expected causal pathways for each policy. The models included short-term outcomes (such as policy awareness, compliance and social cognitive outcomes), intermediate outcomes (such as changes in smoking behaviour) and long-term outcomes (such as mortality, morbidity and health service usage). The use of logic models enables transparent and theory-based planning of evaluation analyses and should be encouraged in the evaluation of tobacco control policy, as well as other areas of public health.

Context and Implications Document for: Using evidence‐informed logic models to bridge methods in educational evaluation

Designs that combine differing forms of data are increasingly used to structure educational evaluation studies, for a variety of reasons. In particular, using combinations of methods can help improve understanding and enable better interpretation of findings from evaluations with a variety of purposes including impact, pilot and scale‐up evaluations, all of which are considered in this paper. The use of logic models as visual representations that lay out the steps from inputs to outcomes of programmes has become widespread as a tool for designing educational evaluations, especially as they have been promoted by policy makers and funders including the Education Endowment Foundation (EEF) in England. Yet, the use of logic models in educational evaluations has not been given due attention as a way of providing robust representation of the intervention being evaluated and for interpreting evaluation findings. The paper reflects on practical and theoretical implications of critical literature on logic models focusing particularly on issues of implementation logic, causal mechanisms, context and complexity. The paper uses two EEF evaluations to illustrate how these issues can be addressed and also to present a new framework for evidence‐based logic models thatdraws out a set of key issues to address in future evaluations that use logic models.

The international knowledge base for new care models relevant to primary care-led integrated models: a realist synthesis

BackgroundThe Multispecialty Community Provider (MCP) model was introduced to the NHS as a primary care-led, community-based integrated care model to provide better quality, experience and value for local populations.ObjectivesThe three main objectives were to (1) articulate the underlying programme theories for the MCP model of care; (2) identify sources of theoretical, empirical and practice evidence to test the programme theories; and (3) explain how mechanisms used in different contexts contribute to outcomes and process variables.DesignThere were three main phases: (1) identification of programme theories from logic models of MCP vanguards, prioritising key theories for investigation; (2) appraisal, extraction and analysis of evidence against a best-fit framework; and (3) realist reviews of prioritised theory components and maps of remaining theory components.Main outcome measuresThe quadruple aim outcomes addressed population health, cost-effectiveness, patient experience and staff experience.Data sourcesSearches of electronic databases with forward- and backward-citation tracking, identifying research-based evidence and practice-derived evidence.Review methodsA realist synthesis was used to identify, test and refine the following programme theory components: (1) community-based, co-ordinated care is more accessible; (2) place-based contracting and payment systems incentivise shared accountability; and (3) fostering relational behaviours builds resilience within communities.ResultsDelivery of a MCP model requires professional and service user engagement, which is dependent on building trust and empowerment. These are generated if values and incentives for new ways of working are aligned and there are opportunities for training and development. Together, these can facilitate accountability at the individual, community and system levels. The evidence base relating to these theory components was, for the most part, limited by initiatives that are relatively new or not formally evaluated. Support for the programme theory components varies, with moderate support for enhanced primary care and community involvement in care, and relatively weak support for new contracting models.Strengths and limitationsThe project benefited from a close relationship with national and local MCP leads, reflecting the value of the proximity of the research team to decision-makers. Our use of logic models to identify theories of change could present a relatively static position for what is a dynamic programme of change.ConclusionsMultispecialty Community Providers can be described as complex adaptive systems (CASs) and, as such, connectivity, feedback loops, system learning and adaptation of CASs play a critical role in their design. Implementation can be further reinforced by paying attention to contextual factors that influence behaviour change, in order to support more integrated working.Future workA set of evidence-derived ‘key ingredients’ has been compiled to inform the design and delivery of future iterations of population health-based models of care. Suggested priorities for future research include the impact of enhanced primary care on the workforce, the effects of longer-term contracts on sustainability and capacity, the conditions needed for successful continuous improvement and learning, the role of carers in patient empowerment and how community participation might contribute to community resilience.Study registrationThis study is registered as PROSPERO CRD42016039552.FundingThe National Institute for Health Research Health Services and Delivery Research programme.

Use of logical models for proving infeasibility in term rewriting

Given a (Conditional) Rewrite System R and terms s and t, we consider the following problem: is there a substitution σ instantiating the variables in s and t such that the reachability testσ(s)→R⁎σ(t) succeeds? If such a substitution does not exist, we say that the problem is infeasible; otherwise, we call it feasible. Similarly, we can consider reducibility, involving a single rewriting step. In term rewriting, a number of important problems involve such infeasibility tests (e.g., confluence and termination analysis). We show how to recast infeasibility tests into the problem of finding a model of a set of (first-order) sentences representing the operational semantics of R together with some additional sentences representing the considered property which is formulated as an infeasibility test.

Origins of Cancer Disparities in Young Adults: Logic Models to Guide Research

Cancer disparities research is motivated by persistent inequities in cancer outcomes by race, ethnicity, social status, neighborhood location, and other subpopulation characteristics. These inequities have proven to be persistent and difficult to alter. Part of the reason for the frustration with slow progress is a lack of appreciation of the long-term nature of the undertaking. It is highly likely that the disparities observed in adulthood find their origins in young adulthood, if not earlier. A long-term perspective is needed, recognizing that successes may take many years to realize. This commentary presents the experience of the Center for Health And Risk in Minority youth and young adults project, which is a comprehensive center of excellence funded by the National Institute of Minority Health and Health Disparities for addressing disparities and chronic disease prevention in minority youth and young adults. The use of logic models is advanced for this kind of research because they can provide a theory of change and illustrate the course of knowledge translation over time. Logic models for cancer disparities research can place individual project activities in a realistic context that at one time indicates what is possible and reasonable to expect during the duration of a typical research project as well as the future steps that need to be taken on the way to an expected ultimate impact on cancer inequities.

Disability Policy Evaluation: Combining Logic Models and Systems Thinking

Policy evaluation focuses on the assessment of policy-related personal, family, and societal changes or benefits that follow as a result of the interventions, services, and supports provided to those persons to whom the policy is directed. This article describes a systematic approach to policy evaluation based on an evaluation framework and an evaluation process that combine the use of logic models and systems thinking. The article also includes an example of how the framework and process have recently been used in policy development and evaluation in Flanders (Belgium), as well as four policy evaluation guidelines based on relevant published literature.

Physical principles for DNA tile self-assembly.

DNA tiles provide a promising technique for assembling structures with nanoscale resolution through self-assembly by basic interactions rather than top-down assembly of individual structures. Tile systems can be programmed to grow based on logical rules, allowing for a small number of tile types to assemble large, complex assemblies that can retain nanoscale resolution. Such algorithmic systems can even assemble different structures using the same tiles, based on inputs that seed the growth. While programming and theoretical analysis of tile self-assembly often makes use of abstract logical models of growth, experimentally implemented systems are governed by nanoscale physical processes that can lead to very different behavior, more accurately modeled by taking into account the thermodynamics and kinetics of tile attachment and detachment in solution. This review discusses the relationships between more abstract and more physically realistic tile assembly models. A central concern is how consideration of model differences enables the design of tile systems that robustly exhibit the desired abstract behavior in realistic physical models and in experimental implementations. Conversely, we identify situations where self-assembly in abstract models can not be well-approximated by physically realistic models, putting constraints on physical relevance of the abstract models. To facilitate the discussion, we introduce a unified model of tile self-assembly that clarifies the relationships between several well-studied models in the literature. Throughout, we highlight open questions regarding the physical principles for DNA tile self-assembly.

‘Modeling’ youth work: logic models, neoliberalism, and community praxis

This paper examines the use of logic models in the development of community initiatives within the AmeriCorps program. AmeriCorps is the civilian national service programme in the U.S., operating as a grants programme to local governments and not-for-profit organisations and providing low-cost labour to address pressing issues of social inequality. Logic models are a popular tool in programme planning, which graphically illustrates the relationship between a programme’s goals, resources expenditures and expected outcomes. Drawing from institutional ethnographic data, this paper examines how the use of logic models within the programme planning process create particular modes of defining and intervening in the achievement gap, thus shaping interventions by youth workers at the level of individual behaviour and obscuring structural roots forms of inequality. It is argued that in such ‘technologies’ of neoliberalism, we can understand how neoliberal public policy works to influence interventions at the community level.

Using logic models to enhance the methodological quality of primary health-care interventions: guidance from an intervention to promote nutrition care by general practitioners and practice nurses.

The methodological designs underpinning many primary health-care interventions are not rigorous. Logic models can be used to support intervention planning, implementation and evaluation in the primary health-care setting. Logic models provide a systematic and visual way of facilitating shared understanding of the rationale for the intervention, the planned activities, expected outcomes, evaluation strategy and required resources. This article provides guidance for primary health-care practitioners and researchers on the use of logic models for enhancing methodological rigour of interventions. The article outlines the recommended steps in developing a logic model using the 'NutriCare' intervention as an example. The 'NutriCare' intervention is based in the Australian primary health-care setting and promotes nutrition care by general practitioners and practice nurses. The recommended approach involves canvassing the views of all stakeholders who have valuable and informed opinions about the planned project. The following four targeted, iterative steps are recommended: (1) confirm situation, intervention aim and target population; (2) document expected outcomes and outputs of the intervention; (3) identify and describe assumptions, external factors and inputs; and (4) confirm intervention components. Over a period of 2 months, three primary health-care researchers and one health-services consultant led the collaborative development of the 'NutriCare' logic model. Primary health-care practitioners and researchers are encouraged to develop a logic model when planning interventions to maximise the methodological rigour of studies, confirm that data required to answer the question are captured and ensure that the intervention meets the project goals.

Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews.

BackgroundLogic models are becoming an increasingly common feature of systematic reviews, as is the use of programme theory more generally in systematic reviewing. Logic models offer a framework to help reviewers to ‘think’ conceptually at various points during the review, and can be a useful tool in defining study inclusion and exclusion criteria, guiding the search strategy, identifying relevant outcomes, identifying mediating and moderating factors, and communicating review findings.Methods and FindingsIn this paper we critique the use of logic models in systematic reviews and protocols drawn from two databases representing reviews of health interventions and international development interventions. Programme theory featured only in a minority of the reviews and protocols included. Despite drawing from different disciplinary traditions, reviews and protocols from both sources shared several limitations in their use of logic models and theories of change, and these were used almost unanimously to solely depict pictorially the way in which the intervention worked. Logic models and theories of change were consequently rarely used to communicate the findings of the review.ConclusionsLogic models have the potential to be an aid integral throughout the systematic reviewing process. The absence of good practice around their use and development may be one reason for the apparent limited utility of logic models in many existing systematic reviews. These concerns are addressed in the second half of this paper, where we offer a set of principles in the use of logic models and an example of how we constructed a logic model for a review of school-based asthma interventions.

Planning and evaluating population interventions to reduce noncommunicable disease risk - reconciling complexity and scientific rigour?

Noncommunicable diseases (NCDs) are the leading global causes of morbidity and mortality. It is important to develop and deliver effective NCD prevention programs, but these have been difficult to evaluate. Technical approaches differ, with academic researchers, practitioners and policy makers each bringing different perspectives and priorities to the task of NCD program evaluation. Epidemiologically defined hierarchies of research evidence give preference to evaluation methods that are often unsuitable for assessing complex NCD prevention interventions. This may lead to interventions that provide the 'right answer to the wrong question', or to evaluation data that are insufficient to inform NCD prevention efforts. This paper recommends a set of standardised stages in the planning, development and evaluation of NCD prevention programs, including the use of logic models, the expanded use of process evaluation to better understand and record the context for implementation, and the use of appropriate research designs for assessing the impact of both subcomponents and the whole program. NCD prevention agencies and academic stakeholders need to recognise the limitations of established evaluation designs and support greater flexibility in the application of evaluation methods that are fit for purpose in describing the stages in NCD programs. This involves assessing policy development and implementation, measuring intermediate indicators, using mixed methods of evaluation, and employing population surveillance systems to assess long-term outcomes.

Understanding interventions for improving routine immunization coverage in children in low- and middle-income countries: a systematic review protocol

BackgroundVirtually all low- and middle-income countries are dependent on the World Health Organization’s Expanded Program on Immunization for delivery of vaccines to children. The Expanded Program on Immunization delivers routine immunization services from health facilities free of charge. Understanding interventions for improving immunization coverage remains key in achieving universal childhood immunization.MethodsWe will conduct a systematic review that aims to assess the effectiveness of the full range of potential interventions to improve routine immunization coverage in children in low- and middle-income countries. We will include intervention studies, as well as observational studies. We will search the Cochrane Database of Systematic Reviews (CDSR), Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, electronic databases for eligible studies published by 31 August 2013. At least two authors will independently screen search outputs, select studies, extract data and assess the risk of bias (using separate criteria for interventions and observational studies); resolving any disagreements by discussion and consensus. The use of logic models and the Cochrane Complexity Matrix will be explored in order to better understand and contextualize studies. We will express the result of each study as a risk ratio with its corresponding 95% confidence intervals for dichotomous data, or mean difference with its standard deviation for continuous data. We will conduct meta-analysis for the same type of participants, interventions, study designs, and outcome measures where homogeneity of data allows. Use of harvest plots may be explored as an alternative. Heterogeneity will be assessed using the χ2 test of heterogeneity, and quantified using the I2 statistic. This protocol has not been registered with PROSPERO.DiscussionThis review will allow us to document evidence across a broad range of intervention types for improving routine immunization coverage in children and also distinguish between those that are well supported by evidence (to direct policy recommendations) and those that are not well supported (to direct research agenda).

Measuring Impacts from Public Health Research

BACKGROUND: The importance of measuring progress towards goals was recently highlighted in the Gates Foundation Annual Letter[1]. Public health researchers sometimes struggle to make direct connections between their activities and interventions with longer term public health impact. The Evaluation Metrics Manual[2], a resource developed by the National Institute of Environmental Health Sciences (NIEHS), can help researchers articulate their goals and develop tangible metrics of progress toward their achievement. This paper will review the Manual, which promotes the use of logic models to demonstrate linkages among research project activities, outputs, and impacts. Particular attention will be given to using logic models to develop specific metrics of public health impact that go beyond typical measures such as counts of peer reviewed publications and journal impact factors. AIMS and METHODS: NIEHS grantees asked for help in understanding and documenting the impacts and outcomes of their work. In response, we developed the Manual based on environmental public health literature, interviews with grantees and reviews of their projects. CONCLUSION: Good measurement is critical for documenting impact. Researchers and science managers benefit from better measurement and evaluation approaches in that they are better able to: identify and publicize program achievements; determine whether a project worked and why (or why not); identify areas for improvement; describe expenditures; justify a need for additional funding; and inform policy and other decision making. [1] Gates, Bill. (2013). Annual Letter from the Bill & Melinda Gates Foundation. http://annualletter.gatesfoundation.org/ [2] NIEHS (2012). Partnerships for Environmental Public Health Evaluation Metrics Manual. http://www.niehs.nih.gov/pephmetrics