Formative Evaluation Approach Research Articles

Islamic content-based learning: Kindergarten illustrated narrative song "Lulu and Tom"

Music education during early life is crucial and has beneficial impacts on the development and maturation of children during this formative period. Music profoundly impacts young children as it fosters their cognitive capacities, language proficiency, motor skills, and imaginative thinking. Teachers must cultivate kids' gross motor abilities by providing opportunities for them to engage in movement activities and songs. This will enable children to experience movement and directly enhance their overall movement proficiency. This research aims to create songs inspired by Islamic illustrated stories that have been demonstrated to be valid and practical. The utilized development model is 4D, incorporating a Tessmer formative evaluation approach that encompasses needs analysis, product design, and assessment stages. The evaluation stages involve self-evaluation, expert review, one-to-one evaluation, and small-group evaluation. The data analysis technique employs walkthroughs, observations, and checklists. The research findings indicate that the average product and media validity is very valid. Three children were assessed individually during the one-to-one evaluation stage, whereas nine children were assessed together during the small group evaluation stage. The averages for the convenient category were derived from both stages. A practical method to cultivate a joyful ambiance during recreational pursuits is to engage in the auditory experience of music and singing. Songs can enhance children's language development, making the learning process easier. Thus, developing songs based on Islamic illustrated stories is pronounced valid, practical, and fit for use.

Evaluating transformative innovation policy in a formative way: Insights from Vinnova’s food mission experiment

Abstract This article presents the insights from an evaluation of a transformative policy experiment, in the food domain, conducted at the Swedish Agency of Innovation (Vinnova). To be consistent with the principles and objectives of these policies, it was necessary to implement a formative evaluation approach developed in the Transformative Innovation Policy Consortium (TIPC), that is supportive of experimental policies designed for highly complex and uncertain environments. The article presents the main characteristics of this approach and its differences and similarities with other proposals to assess transformative innovation policies. Next, the article describes how the evaluation was conducted in Vinnova during a 2-year engagement. Subsequently, the main insights gleaned from the process are highlighted in terms of (1) understanding the expectations of actors; (2) what has been learned by doing, from a technical and organizational point of view; and (3) the constraints that emerged during the engagement. It is then argued that the TIPC- formative evaluation approach challenges the role of evaluators, stretching them, and requires different attitudes from those that are common in evaluation practice.

Psychological profiles of COVID vaccine-hesitant individuals and implications for vaccine message design strategies

COVID-19 has caused tremendous consequences in the U.S., and combating the pandemic requires a significant number of Americans to receive COVID-19 vaccines. Guided by prominent health communication theories, this project took a formative evaluation approach and employed a national sample (N = 1041) in the U.S. to explore the potential differences between vaccine-inclined vs. -hesitant individuals and to generate profiles of hesitant individuals as the foundation for audience segmentation and message targeting. Five distinct profiles emerged in the sample. Characteristics of each profile were described, and appropriate messaging strategies were identified to target each group. Theoretical and practical implications were discussed.

Evaluation design of the patient-centred pathways of early palliative care, supportive ecosystems and appraisal standard (InAdvance): a randomised controlled trial

BackgroundPalliative care aims to contribute to pain relief, improvement with regard to symptoms and enhancement of health-related quality of life (HRQoL) of patients with chronic conditions. Most of the palliative care protocols, programmes and units are predominantly focused on patients with cancer and their specific needs. Patients with non-cancer chronic conditions may also have significantly impaired HRQoL and poor survival, but do not yet receive appropriate and holistic care. The traditional focus of palliative care has been at the end-of-life stages instead of the relatively early phases of serious chronic conditions. The ‘Patient-centred pathways of early palliative care, supportive ecosystems and appraisal standard’ (InAdvance) project implements and evaluates early palliative care in the daily clinical routine addressing patients with complex chronic conditions in the evolution towards advanced stages. The objective of the current study is to evaluate the acceptability, feasibility, effectiveness and cost-effectiveness of this novel model of palliative care in the relatively early phases in patients with chronic conditions.MethodsIn this study, a single blind randomised controlled trial design will be employed. A total of 320 participants (80 in each study site and 4 sites in total) will be randomised on a 1:1 basis to the Palliative Care Needs Assessment (PCNA) arm or the Care-as-Usual arm. This study includes a formative evaluation approach as well as a cost-effectiveness analysis with a within-trial horizon. Study outcomes will be assessed at baseline, 6 weeks, 6 months, 12 months and 18 months after the implementation of the interventions. Study outcomes include HRQoL, intensity of symptoms, functional status, emotional distress, caregiving burden, perceived quality of care, adherence to treatment, feasibility, acceptability, and appropriateness of the intervention, intervention costs, other healthcare costs and informal care costs.DiscussionThe InAdvance project will evaluate the effect of the implementation of the PCNA intervention on the target population in terms of effectiveness and cost-effectiveness in four European settings. The evidence of the project will provide step-wise guidance to contribute an increased evidence base for policy recommendations and clinical guidelines, in an effort to augment the supportive ecosystem for palliative care.Trial registrationISRCTN, ISRCTN24825698. Registered 17/12/2020.

How Health Care Practitioners experience emergencies at Primary Health Care facilities - Kinks in the chain of survival.

BackgroundThe 72nd World Health Assembly has recognised that emergency care at primary health care level is vital for reducing overall mortality and disability. The system of emergency care at this level is affected by various external factors. Little is known about these factors and how they shape the experiences of health care practitioners dealing with medical emergencies in Primary Health Care (PHC) settings. The objective of the study was to explore the experiences of health care practitioners in dealing with emergencies in PHC facilities in the Gauteng province of South Africa.MethodsA qualitative formative evaluation approach was used. Data were collected using semi structured interviews and analysed using qualitative content analysis to describe the experiences of health care practitioners dealing with emergencies at a primary health care level. Participants included health care practitioners from various levels of the district health system.ResultsMajor themes that emerged explored challenges faced by health care practitioners, the referral system and influential policy such as the ideal clinic movement.

Visual analysis of blow molding machine multivariate time series data

The recent development in the data analytics field provides a boost in production for modern industries. Small-sized factories intend to take full advantage of the data collected by sensors used in their machinery. The ultimate goal is to minimize cost and maximize quality, resulting in an increase in profit. In collaboration with domain experts, we implemented a data visualization tool to enable decision-makers in a plastic factory to improve their production process. The tool is an interactive dashboard with multiple coordinated views supporting the exploration from both local and global perspectives. In summary, we investigate three different aspects: methods for preprocessing multivariate time series data, clustering approaches for the already refined data, and visualization techniques that aid domain experts in gaining insights into the different stages of the production process. Here we present our ongoing results grounded in a human-centered development process. We adopt a formative evaluation approach to continuously upgrade our dashboard design that eventually meets partners’ requirements and follows the best practices within the field. We also conducted a case study with a domain expert to validate the potential application of the tool in the real-life context. Finally, we assessed the usability and usefulness of the tool with a two-layer summative evaluation that showed encouraging results.Graphical

Formative Evaluation of College Students’ Online English Learning Based on Learning Behavior Analysis

To measure the achievements and progress of college students in online learning, online learning platforms and teachers must pay attention to the formative evaluation of the learning process. The relevant data should be fully utilized to analyze the online English learning behavior of college students, such that online learning platforms and teachers can make formative evaluation of the students’ online learning. However, the existing studies on formative evaluation are mostly theoretical. To solve the problem, this paper explores the formative evaluation of college students’ online English learning based on learning behavior analysis. Firstly, the density-based spatial clustering of applications with noise (DBSCAN) was adopted to analyze the data samples of college students’ online English learning behavior, the evaluation indices were selected for the formative evaluation of the said behavior, and the index weighting method was explained in details. Next, the school precaution function of online English learning was realized through the graph structure data prediction of students’ online learning behavior. Based on the proposed graph neural network, the clustering Euclidean distance weight was introduced to measure the similarity between two nodes. In addition, the weight update process was illustrated for the distance weight-based attention mechanism. The proposed formative evaluation approach was proved effective through experiments.

EVALUASI IMPLEMENTASI PROGRAM INTERVENSI BERBASIS MASYARAKAT DI WILAYAH JAKARTA UTARA

The focus of this research is to evaluate how the implementation of IBM program activities and services by BNN Kota Jakarta Utara and Recovery Agency officers in the North Jakarta area. The indicator seen is how the conformity between the program being run with the expected results as a benchmark of the success of the program in the field. This study was conducted on 6 (six) villages in the North Jakarta area as villages that have criteria to conduct P4GN programs, the research was conducted in 2021. Qualitative research methods are used to determine the quality or quality of service from a product-based perspective, with a formative evaluation approach or program evaluation. The conclusions of this study are used for the submission of means in accordance with eligibility standards. Product evaluation activities include program operational goal setting activities, measurement criteria that have been achieved, comparing them between the field realities of goal formulation, and rational interpretation.

Evaluating Petrophysical Properties and Volumetrics Uncertainties of Sand Injectite Reservoirs – Norwegian North Sea

Sand injectites on the Norwegian Continental Shelf have proven their commercial significance. Some are already producing, e.g., Volund, Viper, Balder, Ringhorne, and Kobra Fields, while others such as in production licenses (PL) 340 and 869 have recently been discovered and appraised. Extensive literature on the geology of sand injectites has been published (e.g., Jenssen et al., 1993; Jolly and Lonergan, 2002; Huuse et al., 2003; Hurst et al., 2005). However, few references are available on the petrophysical and geophysical aspects of sand injectite reservoirs. This paper discusses the petrophysical properties of sand injectite facies, dykes, sills, and brecciated sands, along with their identification from seismic data. A perception that volumetrics of sand injectite reservoirs cannot be reliably evaluated is assessed. Sand injectites in PL 340 and 869 were interpreted as remobilized sands from the Hermod and Heimdal Formations of Paleocene age injected into the overlying Balder Formation and Hordaland Group mudstones of Eocene age. The mudstones acted as a seal, forming an intrusive stratigraphic trap. The trap geometry varied locally depending upon the dyke and sill geometries of the sandstone. Dykes had large vertical reach with the corresponding high-hydrocarbon column, while sills had low-vertical relief with large lateral extent. Intervals of brecciated sands were also observed within the injectite complex, especially where sands were thin. These brecciated sands contained large amounts of angular mudstone clasts of different dimensions suspended in an overall sandy matrix. Close examination of cored dykes made it possible to observe this, while it might not be as obvious when looking at bulk well logs. Petrophysical-log responses for clean sills and dykes behaved the same way as they would in a clean sandstone reservoir. If sills and dykes were very thin, they would also risk not being counted as net or pay (Suau et al., 1984; Dromgoole et al., 2000; Flølo et al., 2000;). Such errors can impact in-place volumes in a significant way. Sills appeared as blocky clean sand on logs, but it was difficult to differentiate a dyke from a sill or thin sands using logs. Dykes are high-angle features and are identified either by core studies or borehole images when intersected by a well or, if large enough, observable on seismic. Brecciated sand intervals appeared with cm-to-dm-scale mudstone clasts suspended in sand with approximately 40 to 60% net to gross. Log responses over these intervals indicated shaly sand or thin sands. Resistivity and thermal neutron porosity logs were highly affected by the shale clasts. For this reason, a fractional net/gross interpretation technique was used to evaluate the sand content and hydrocarbon pore volume. To further verify these results, they were compared to observations directly on the core. To qualify to what extent petrophysical logs and interpreted products thereof can be relied on to evaluate hydrocarbon volumes of sand injectite reservoirs, a high-resolution petrophysical interpretation was generated using a computerized tomography (CT) scanned core image. Core image sand counting and image-derived high-resolution bulk density logs with shale-corrected resistivity were used. Results of this high-resolution interpretation featured an excellent match with routine core analysis data and manual core observations in the core laboratory. The fractional net/gross method used is the modified Thomas-Stieber method (Johansen et al., 2018). Its results compared well to the high-resolution CT-scan image results and better evaluated the hydrocarbon pore volume of sand facie compared to the conventional bulk formation evaluation approach. This result confirms that the Thomas-Stieber method can be used for brecciated rocks, which leads to some useful recommendations on how to best log and perform a petrophysical evaluation in such reservoirs.

Design and Development of Matter Module with the Integration of Computational Thinking by Employing ADDIE Model

This study aims to explore the design and development process of the Matter module for secondary school students. The needs analysis shows a similar perception between students and teachers who described the matter topic as complex due to the abstract nature of the Matter topic. This module emphasises the use of animation in visualising particles in different matter states. The module was designed to make learning more active and meaningful by integrating computational thinking skills through cooperative and project-based learning. Through this approach, students are actively involved in building animation projects to visualise the movement of particles by applying computational thinking concepts (abstraction, decomposition, generalisation, algorithm, and evaluation) as they engage with programming in Scratch. Thus, to systematically design and develop the Matter module, the ADDIE model was employed. The design and development of the module comprise five phases, Analysis, Design, Development, Implementation, and Evaluation. The module was evaluated by experts using a formative evaluation approach. The results show that the validity of the Matter module is high and can support the integration of computational thinking skills.

A principles-focused evaluation of community mental health short-term case management

Principles-focused evaluations allow evaluators to appraise each principle that guides an organization or program. This study completed a principles-focused evaluation of a new community mental health intervention called Short Term Case Management (STCM) in Toronto, Canada. STCM is a time limited intervention for clients to address unmet needs and personalized goals over 3 months. Findings showcase that a principles-focused evaluation, assessing whether program principles are guiding, useful, inspiring, developmental and/or evaluable (GUIDE), is a practical formative evaluation approach. Specifically, offering an understanding of a novel intervention, including its key components of assessment and planning, support plan implementation and evaluation and care transitions. Findings also highlight that STCM may work best for those clients ready to participate in achieving their own goals. Future research should explore how best to apply the GUIDE framework to complex interventions, including multiple principles, to increase evaluation feasibility and focus.

Qualitative Evaluation of the Africa Malaria Task Force Symposia Effectiveness on African Partner Nations' Malaria Vector Management Capacity.

Creating health alliances to build meaningful networks is critical to combating regional and global burdens of disease. These alliances work by uniting support for elimination efforts through cooperative engagement at the national and international levels. The reduction in malaria-related morbidity and mortality in Africa since 2001 is in part because of investments of international organizations and governments in national level malaria control and prevention-related programs and research. Investment in malaria reduction networks has contributed to this success by strengthening support to overcome the conditions that restrict or prevent change through local laboratory and epidemiological capacity building, thereby resulting in a decrease in burden of disease, increase in economic prosperity, and improvements in stability worldwide.The reformation of local military efforts to combat disease through incorporation into health security alliance networks by the provision of training and financial support is key to reinforcing this success at the national level. One such example of this is the U.S. Africa Command (USAFRICOM) established and partner nation (PN) led Africa Malaria Task Force (AMTF) program. USAFRICOM's mission for AMTF as a health security alliance was to develop sustainable African-led malaria diagnostic, vector surveillance and control capabilities, and to increase collaborations among AMTF military PNs by enabling national and international-level network capacity. The overall objective of this evaluation of the AMTF program was to assess the implementation effectiveness of two AMTF supported symposia in improving the individual, organizational, and institutional diagnostic capacity of the Ghanaian, Ivorian, and Senegalese armed forces. The specific aims of this process evaluation were to identify procedural, contextual, and capacity-related factors that influence the effect of the symposia events on laboratory capabilities at the individual, organizational, and institutional levels through a two-phased formative evaluation approach utilizing qualitative methodologies. Findings indicate that the AMTF program symposia positively encouraged malaria prevention and control efforts in all three countries included in the evaluation. Conclusions suggest that learning about the different types of malaria prevention and control efforts underway in other countries helped to contextualize the burden of malaria-related morbidity and mortality not only within their respective countries, but also across the African continent. Participation in the AMTF Symposia and Key Leader Events helped military leadership clarify the purpose and intention of their military objectives related to systemic malaria prevention and control while purposefully contributing to national land international-level malaria reduction capacity.

Formation Evaluation Workflow Assesses Mass-Transport Complex Reservoirs

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 203016, “Formation Evaluation in Mass-Transport Complex Reservoirs,” by Ulises Bustos, Schlumberger; Carlos Duran, Petróleos Sudamericanos; and Alvaro Chapellin, Schlumberger, et al., prepared for the 2020 Abu Dhabi International Petroleum Exhibition and Conference, Abu Dhabi, held virtually from 9-12 November. The paper has not been peer reviewed. Mass-transport deposits (MTDs) are sedimentary, stratigraphic successions remobilized after initial deposition but before substantial lithification and transported downslope by gravitational processes as non-Newtonian rheological units. In the complete paper, the authors present an openhole advanced formation-evaluation approach that enables assessment of tight-matrix and natural-fracture systems at a level not previously accomplished in these types of geological formations. Introduction The considered wildcat project by Petroleos Sudamericanos is in the Lower Magdalena Valley hydrocarbon province in Colombia. From a stratigraphic point of view, the targets belong to tertiary deposits from the lower Neogene. Gravity-driven processes are complex and include creep, slide, slump, debris flow, and multiphase granular flows. The remobilized sedimentary deposits resulting from these processes are called MTDs or mass-transport complexes (MTCs) and are the main target job. Overlaying a crystalline basement, four MTC cycles were identified (although the present work only covers three MTC cycles) deposited in shallow marine environments. Each cycle consists of quartzite; phyllite; and schistose metamorphic rocks, largely gneiss with strong milonitization effects and foliated tremolitic marble in the top of the sequence. A summary of these types of rock is provided in the complete paper. The non-Archie nature of these rocks represents a challenge for formation evaluation. For reducing these uncertainties, a volumetric model with spectros copy dry-weight elements and nuclears was created that enabled solution of the total porosity, which was then benchmarked against the lithology-independent total nuclear magnetic resonance (NMR) porosity. The saturation computation was achieved with the fast neutron cross section method (FNXS) for the gas component and with total carbon for the liquid hydrocarbon fraction. The natural fracture system was analyzed with borehole image logs and with radial sonic-based dispersion analysis. The integration of matrix and natural-fracture assessment provided a robust formation evaluation that enabled identification of the main interest zones across the MTC cycles. Multifunction Spectroscopy for Matrix Analysis The multifunction spectroscopy tool is 1.72 in. in diameter and 18.3 ft long, consisting of a pulsed neutron generator and four detectors. The first detector is the compact neutron monitor, sensitive to fast neutrons and located adjacent to the high-output pulsed neutron source, measuring neutron output with high accuracy and precision. The second (near) and third (far) gamma-ray detectors are used for inelastic and capture spectroscopy measurements. The fourth and farthest-spaced detector (deep) uses an yttrium aluminum perovskite scintillator, which is involved in gas detection and assessment. The nuclear spectroscopy measurement is performed in energy and time domains; both aspects are described in detail in the complete paper.

Risky sexual behaviour amidst predicament of acceptable sexually transmitted infection partner notification modalities: A cross-sectional survey amongst minibus taxi drivers in Gauteng Province, South Africa.

BackgroundSexually transmitted infection patient-initiated partner notification (PN) approach has been implemented over a decade in South Africa, however, use and update by patients has been limited. This study assessed the perceived use of patient-initiated PN by using referral slips and measured the level of acceptability of provider-initiated PN by using short message service (SMS) to the personal mobile phones of sexual partners.MethodsA formative evaluation approach using a quantitative survey amongst 722 minibus taxi drivers in nine major taxi ranks in Tshwane Municipality, Gauteng Province. STATA IC version 13 was used for data analyses.ResultsThe mean age of the participants was 37.2 years (59%) were single, 59.5% had multiple sexual partners, 52.2% did not use a condom during the last sexual act, 42.8% reported inconsistent use of condoms and 65% tested for HIV in the past 12 months. The majority (98.2%) understood the importance of PN, but 51% would prefer telling a sexual partner face to face. Perceived easiness of delivering a PN slip was 69.1%, and 93% would use a PN slip received from a partner. Acceptability of provider-initiated PN by using an SMS was 62.7% and about a third (32.5%) were not in favour of provider-initiated PN by SMS. Twenty four point 7 percent (24.7%) preferred patient-initiated PN and 24.3% preferred provider-initiated PN.ConclusionPreferred patient-initiated and provider-initiated PN was almost equal, hence, the provider-initiated PN should be augmented to support the current patient-initiated PN to increase the overall STI notification.

Mobile-based consortium of parenting resources for low-income and underserved mothers and caregivers: app development, testing and lessons learned

This pilot study used a formative evaluation approach to develop an accessible and low-cost mobile app-based program for low-income mothers to increase access to resources and health information (e.g., effective parenting) and reduce potential negative (e.g., stress, poor nutritional practices) influences. A user-centered formative evaluation method using four sets of focus group discussions was used to gather feedback from low-income mothers in a mid-sized city in the Northeast United States. Our study found that while information and resources provided on our parenting app were considered valuable by low-income mothers, the design and usability issues impacted the perceived attractiveness and usefulness of the platform. The younger mothers wanted functionality similar to popular social media apps, and given their mostly constrained financial situation, they were more engaged and interested in community resources that would help reduce their financial burden, instead of the purely preventative health and parenting information provided through our app.

Evaluasi Formatif: Perencanaan dan Penganggaran Responsif Gender (PPRG) Provinsi Kepulauan Riau

Penelitian ini mengkaji pelaksanaan perencanaan penganggaran responsif gender (PPRG) di lingkungan Pemerintah Provinsi Kepulauan Riau melalui pendekatan evaluasi formatif. Seringkali dipahami bahwa gender hanya memprioritaskan jenis kelamin tertentu. Pemegang dan perumus kebijakan dalam hal ini Pemerintah Provinsi Kepulauan Riau melakukan berbagai strategi agar para stakeholder tidak bias gender terutama para pengambil keputusan. Salah satu konsep yang mendukung PPRG adalah Gender equality yang merupakan salah satu indikator capaian dalam SDG’s. Pengintegrasian agenda PPRG kedalam format anggaran berbasis kinerja (ABK) merupakan bentuk gerakan pemerintah untuk mengejar perwujudan gender equality tersebut sampai pada tingkat lokal. Kajian ini menggunakan pendekatan eksploratif untuk menggali dan menjawab secara lengkap dinamika adaptasi PPRG ke dalam sistem perencanaan dan penganggaran daerah yang diturunkan dalam program dan kegiatan. Upaya Pemerintah Provinsi Kepulauan Riau mengintegrasikan PPRG dengan menggunakan perspektif governance dimana fokus pada aktor yang terlibat dari unsur focal point di unit kerja, akademisi, sektor swasta,organisasi kemasyarakatan hingga lembaga swadaya masyarakat dapat dikatakan berhasil. Ini ditunjukkan dengan Indeks Pembangunan Gender Provinsi Kepulauan Riau yaitu ranking 10 besar tepatnya pada peringkat ke 7 se-Indonesia. Rekomendasi untuk pelaksanaan PPRG dengan memaksimalkan fungsi coaching clinic untuk penyusunan Gender Analysis Pathway (GAP) dan Gender Budget Statement (GBS).

Student Opinions on Task-Based Approach as Formative Evaluation versus Exam-Based Approach as Summative Evaluation in Education

The present study investigated ELT students’ opinions about the convenience of exam-based or task-based approaches for their learning process, concerning three criteria that relate to the effectiveness of task-based and exam-based education: 1. the contribution to the professional and personal development of the students (whether task-based or exam-based applications develop them better); 2. the contribution to their critical and creative skills; and 3. the advantages and disadvantages of task-based and exam-based approaches. Exam-based approach as summative assessment type and task-based approach as formative assessment type were compared in terms of their advantages and disadvantages. Findings indicate that the majority of the students are not satisfied with the summative assessment system. Participants believe that the formative evaluation approach is more appropriate for them. The results propose that the conventional approach, which is based on an exam-focused system in the English Language Teaching Department, should be changed in order to enhance future English teachers’ creative skills as well as their personal and professional development as far as possible.

Closing the Gap: Participatory Formative Evaluation to Reduce Cancer Screening Disparities among Patients with Limited English Proficiency.

Patients with limited English proficiency (LEP) have disproportionately lower rates of cancer screening than English-proficient patients. Given the multifactorial nature of screening disparities, strategies to improve screening rates must address barriers within and outside of the clinic setting. The objectives of this study were to understand local barriers from multiple stakeholder perspectives, to identify potential multilevel intervention approaches, and to mobilize community-engaged intervention decision making and planning. This participatory formative evaluation approach employed needs assessment and user engagement in order to enhance intervention usefulness and relevance. The study took place in several stages and involved clinic and community partners in a small metropolitan area of the Midwest USA. Interviews were conducted with LEP patients (n = 9) who had not completed three recommended screenings (breast, cervical, and colorectal), primary care providers (n = 5), medical interpreter (n = 5), and community members (n = 3). These highlighted multilevel barriers including limited patient understanding of preventive health, time and cost constraints, and variable roles of language interpreters. The literature was also reviewed to identify interventions used with similar populations. Findings from this review suggest that interventions are largely focused on single population groups or address single screening barriers. Finally, a community-academic summit (n = 48 participants) was held to review results and develop recommendations for community and clinic interventions. Findings from this study indicate that it is possible to engage a diverse group of stakeholders in strategies that are responsive to health care providers and patients, including LEP patients from heterogeneous backgrounds.

Evaluating knowledge transfer at the interface between science and society

In view of the global grand challenges, fundamental research institutions are increasingly being asked to provide context for the application of their research findings and to incorporate transdisciplinary forms of knowledge production. But how can the involvement of stakeholders from outside academia be captured and evaluated within the research process? And how can they be engaged in meaningful science-stakeholder dialogue? “Good” processes are a prerequisite for meeting these changing requirements and for ensuring a successful knowledge transfer at the interface between science and society.Societal challenges are increasing on a global scale, requiring an intensification of knowledge transfer (KT) between scientific and societal actors on multiple levels. These transfers bring up new demands regarding the way in which knowledge is produced and transferred, which mechanisms are utilized to ensure the quality of these knowledge interactions, and how such interactions can be evaluated. Capturing and evaluating KT, however, also opens up new reflections about what a meaningful “impact” is and how KTs are shaped and driven. The results presented here were reached by means of a formative and summative evaluation approach, and include an accompanying research effort that aimed to capture central KT processes from start to “finish” by a tailored KT process assessment framework. This framework was applied to the ongoing activities of twelve in-house KT projects (from 2014 to 2017) that were conducted in a fundamental natural science research institution within the field of earth system science. Our findings indicate that, among other things, the continuous assessment of the underlying processes of KT allows for processes and outcomes to be directly influenced, while also providing scope for institutional learning. Better insights into the definition of “societal relevance” may in fact not start with the result, but rather with the question of how the research will be conducted and for whom.

Formative Evaluation and Complex Health Improvement Initiatives: A Learning System to Improve Theory, Implementation, Support, and Evaluation

Sustainable community health improvement often requires the implementation of complex interventions in complex systems. Drawing from the Four Keys to Success frame (theory, implementation, support, and evaluation), this article describes how we used a formative evaluation approach to foster a learning system capable of monitoring and addressing emerging community needs within the Spreading Community Accelerators Through Learning and Evaluation (SCALE) initiative—a national capacity-building effort to support 24 community coalitions’ progress toward a Culture of Health. The formative evaluation approach resulted in critical advancements to the theory, implementation, and nature of supports provided in SCALE. These improvements enabled the SCALE evaluation team to shift from the initial focus on program implementation issues to a greater emphasis on downstream factors (community-level outcomes). The ability of formative evaluation to grapple with the emerging challenges of implementing complex interventions in complex systems makes it particularly valuable for community health improvement initiatives.