Formative Evaluation Study Research Articles

Transfer learning for well logging formation evaluation using similarity weights

Machine learning has been widely applied in well logging formation evaluation studies. However, several challenges negatively impacted the generalization capabilities of machine learning models in practical implementations, such as the mismatch of data domain between training and testing datasets, imbalances among sample categories, and inadequate representation of data model. These issues have led to substantial insufficient identification for reservoir and significant deviations in subsequent evaluations. To improve the transferability of machine learning models within limited sample sets, this study proposes a weight transfer learning framework based on the similarity of the labels. The similarity weighting method includes both hard weights and soft weights. By evaluating the similarity between test and training sets of logging data, the similarity results are used to estimate the weights of training samples, thereby optimizing the model learning process. We develop a double experts' network and a bidirectional gated neural network based on hierarchical attention and multi-head attention (BiGRU-MHSA) for well logs reconstruction and lithofacies classification tasks. Oil field data results for the shale strata in the Gulong area of the Songliao Basin of China indicate that the double experts’ network model performs well in curve reconstruction tasks. However, it may not be effective in lithofacies classification tasks, while BiGRU-MHSA performs well in that area. In the study of constructing large-scale well logging processing and formation interpretation models, it is maybe more beneficial by employing different expert models for combined evaluations. In addition, although the improvement is limited, hard or soft weighting methods is better than unweighted (i.e., average-weighted) in significantly different adjacent wells. The code and data are open and available for subsequent studies on other lithofacies layers.

Evaluation of an online arts-based platform to support the health and well-being of older adults during the COVID-19 pandemic: a cross-sectional survey

ObjectivesThe objective of this study was to conduct a formative evaluation of the Art Your Service (AYS) arts-based program to determine the program’s potential for improving the social and physical well-being of older adults during the COVID-19 pandemic.Design, settings and participantsAn online questionnaire was administered to the AYS members who consented to be invited to participate in the study. Questionnaire items consisted of a Likert scale and open-ended questions delivered using an online platform (SurveyMonkey). Participants provided feedback on their perceptions and experiences of the AYS program, such as its impact on their health and well-being during the COVID-19 pandemic, the benefits and challenges of participating, and any suggestions for program improvement.Outcome measures and analysisQuantitative data were analyzed using descriptive statistics (frequencies, means with standard deviations), and open-ended questions (qualitative data) were analyzed using content analysis. Outcomes included participant demographics, perceptions about the program, usability (System Usability Scale [SUS]), eHealth literacy (eHealth Literacy Scale), and social isolation (Lubben Social Network Scale; LSNS-6).ResultsProgram participants revealed consistent patterns of their perceptions and experiences about the program, including a high satisfaction rate (95%) and a perceived positive impact on participants’ health and well-being. The program sessions were perceived to be a well-organized, convenient, and safe way to engage with one another socially during the COVID-19 pandemic. The program usability was also perceived to be high (SUS mean score 86.2). Participants felt a sense of connectedness and had reduced feelings of social isolation. Most participants (75%) reported that the program improved their physical health.ConclusionsFindings from this formative evaluation study identified key strengths and opportunities to improve the Art Your Service arts-based program, which can be used to help enhance the program’s functioning and long-term sustainability potential.

Exhibition research and practice at CERN: challenges and learnings of science communication `in the making'

This practice insight paper presents a reflection on a four-year collaboration between science communication practitioners and researchers, using CERN's new education and outreach centre as a case study. The development of interactive exhibitions for this centre was informed by a variety of front-end and formative evaluation studies, from online surveys to exhibit prototype testing. As a multidisciplinary team of exhibition developers and social science researchers, we describe and discuss the challenges of — as well as learnings from — working together. Our experience will be relevant for everyone curious to discover `behind-the-scenes' work of research-informed exhibition development in a large scientific laboratory.

Machine Learning Model for Readmission Prediction of Patients With Heart Failure Based on Electronic Health Records: Protocol for a Quasi-Experimental Study for Impact Assessment.

Care for patients with heart failure (HF) causes a substantial load on health care systems where a prominent challenge is the elevated rate of readmissions within 30 days following initial discharge. Clinical professionals face high levels of uncertainty and subjectivity in the decision-making process on the optimal timing of discharge. Unwanted hospital stays generate costs and cause stress to patients and potentially have an impact on care outcomes. Recent studies have aimed to mitigate the uncertainty by developing and testing risk assessment tools and predictive models to identify patients at risk of readmission, often using novel methods such as machine learning (ML). This study aims to investigate how a developed clinical decision support (CDS) tool alters the decision-making processes of health care professionals in the specific context of discharging patients with HF, and if so, in which ways. Additionally, the aim is to capture the experiences of health care practitioners as they engage with the system's outputs to analyze usability aspects and obtain insights related to future implementation. A quasi-experimental design with randomized crossover assessment will be conducted with health care professionals on HF patients' scenarios in a region located in the South of Sweden. In total, 12 physicians and nurses will be randomized into control and test groups. The groups shall be provided with 20 scenarios of purposefully sampled patients. The clinicians will be asked to take decisions on the next action regarding a patient. The test group will be provided with the 10 scenarios containing patient data from electronic health records and an outcome from an ML-based CDS model on the risk level for readmission of the same patients. The control group will have 10 other scenarios without the CDS model output and containing only the patients' data from electronic medical records. The groups will switch roles for the next 10 scenarios. This study will collect data through interviews and observations. The key outcome measures are decision consistency, decision quality, work efficiency, perceived benefits of using the CDS model, reliability, validity, and confidence in the CDS model outcome, integrability in the routine workflow, ease of use, and intention to use. This study will be carried out in collaboration with Cambio Healthcare Systems. The project is part of the Center for Applied Intelligent Systems Research Health research profile, funded by the Knowledge Foundation (2021-2028). Ethical approval for this study was granted by the Swedish ethical review authority (2022-07287-02). The recruitment process of the clinicians and the patient scenario selection will start in September 2023 and last till March 2024. This study protocol will contribute to the development of future formative evaluation studies to test ML models with clinical professionals. PRR1-10.2196/52744.

Reflected Reality

We propose Reflected Reality: a new dimension for augmented reality that expands the augmented physical space into mirror reflections. By synchronously tracking the physical space in front of the mirror and the reflection behind it using an AR headset and an optional smart mirror component, reflected reality enables novel AR interactions that allow users to use their physical and reflected bodies to find and interact with virtual objects. We propose a design space for AR interaction with mirror reflections, and instantiate it using a prototype system featuring a HoloLens 2 and a smart mirror. We explore the design space along the following dimensions: the user's perspective of input, the spatial frame of reference, and the direction of the mirror space relative to the physical space. Using our prototype, we visualise a use case scenario that traverses the design space to demonstrate its interaction affordances in a practical context. To understand how users perceive the intuitiveness and ease of reflected reality interaction, we conducted an exploratory and a formal user evaluation studies to characterise user performance of AR interaction tasks in reflected reality. We discuss the unique interaction affordances that reflected reality offers, and outline possibilities of its future applications.

Internet-Delivered Cognitive Behavioral Therapy Tailored to Spouses and Significant Others of Public Safety Personnel: Formative Evaluation Study.

Spouses and significant others (SSOs) of public safety personnel (PSP) are affected by the risks and requirements of these occupations. Internet-delivered cognitive behavioral therapy (ICBT) provides a convenient and accessible treatment format that can be tailored to the needs of SSOs of PSP. This study aimed to assess the initial use and client perceptions (eg, likes, helpfulness, and areas for improvement) of a self-guided, transdiagnostic ICBT course designed for Canadian SSOs of PSP and identify opportunities to further tailor ICBT for this group. SSOs were invited to complete a 5-lesson, self-guided, transdiagnostic ICBT course. Descriptive statistics were used to analyze the demographic and clinical characteristics of participants. Content analysis was used to analyze the data from open-ended survey responses and interviews to understand their experiences with ICBT. Clients (N=118) endorsed various mental health concerns (eg, depression, anxiety, posttraumatic stress symptoms, and relationship concerns) with a range of severity levels. Most clients identified as White (110/116, 94.8%) and women (108/116, 91.5%), with a mean age of 42.03 (SD 9.36) years. Of the 26 clients who were interviewed, 89% (23/26) reported believing that ICBT is helpful and 92% (24/26) reported finding at least 1 skill helpful. Clients provided suggestions for course improvements. On the basis of this feedback and quantitative data, changes were made to areas such as the delivery of materials, content, case stories, and timelines. Overall, the results indicated that many SSOs of PSP had positive perceptions of ICBT tailored to their needs and found several aspects of the course helpful, supporting the continued delivery of tailored ICBT to this population. However, there remains a need for continued promotion of the course and outreach to diverse groups of SSOs of PSP. Findings from this formative evaluation provide insight into the unique experiences and needs of SSOs of PSP and provide preliminary evidence for the use of tailored ICBT to support the mental health of this group in Canada.

Management of Addiction of Tobacco Through Ayurveda - A Formative Evaluation Study

According to global cigarette smoking statistics, over a billion people are estimated to be regular smokers, despite knowing the harmful effects of it. This is because they are unable to discontinue the tobacco due to severe addiction. The same thing happens with tobacco when taken in chewing form. The present paper is Literary Research designed based on formative evaluation studies to evaluate the existing phenomena mentioned in Ayurveda and construct the treatment method which can be implemented to treat the hazardous effects of tobacco and to help in its de-addiction. A review of literature is done primarily from three major classical Ayurveda texts mainly Charak Samhita, Sushruta Samhita, and Ashtang Hruday. The references co-relating with inhaling and chewing toxic substances are collected from these classical texts. Treatment methods and herbs used in these references are then compiled, and categorized and a treatment plan is designed for the de-addiction of tobacco. Classical treatments of Ayurveda like Dhumpan, Kaval and Gandush, and Nasya are helpful to reduce urges and cravings for tobacco. Shirodhara and Yoga provide tranquility of mind and reduce stress and anxiety. Panchkarma therapy purifies the body by eliminating accumulated toxins. Rasayan treatment helps to rejuvenate and recover the damage caused by tobacco. It can be concluded that the principles of Ayurveda are useful to quit tobacco addiction and reduce its hazardous effects.

Complex implementation mechanisms in primary care: do physicians’ beliefs about the effectiveness of innovation play a mediating role? Applying a realist inquiry and structural equation modeling approach in a formative evaluation study

BackgroundThe adoption of digital health technologies can improve the quality of care for polypharmacy patients, if the underlying complex implementation mechanisms are better understood. Context effects play a critical role in relation to implementation mechanisms. In primary care research, evidence on the effects of context in the adoption of digital innovation for polypharmacy management is lacking.Study aimThis study aims to identify contextual factors relevant to physician behavior and how they might mediate the adoption process.MethodsThe physicians who participated in this formative evaluation study (n = 218) were part of the intervention group in a cluster-randomized controlled trial (AdAM). The intervention group implemented a digital innovation for clinical decision making in polypharmacy. A three-step methodological approach was used: (1) a realist inquiry approach, which involves the description of a context-mechanism-outcome configuration for the primary care setting; (2) a belief elicitation approach, which involves qualitative content analysis and the development of a quantitative latent contextualized scale; and (3) a mediation analysis using structural equation modeling (SEM) based on quantitative survey data from physicians to assess the mediating role of the contextualized scale (n = 179).ResultsThe key dimensions of a (1) context-mechanism-outcome model were mapped and refined. A (2) latent construct of the physicians’ innovation beliefs related to the effectiveness of polypharmacy management practices was identified. Innovation beliefs play a (3) mediating role between the organizational readiness to implement change (p < 0.01) and the desired behavioral intent of physicians to adopt digital innovation (p < 0.01; R2 = 0.645). Our contextualized model estimated significant mediation, with a relative size of 38% for the mediation effect. Overall, the model demonstrated good fit indices (CFI = 0.985, RMSEA = 0.034).ConclusionPhysician adoption is directly affected by the readiness of primary care organizations for the implementation of change. In addition, the mediation analysis revealed that this relationship is indirectly influenced by primary care physicians’ beliefs regarding the effectiveness of digital innovation. Both individual physician beliefs and practice organizational capacity could be equally prioritized in developing implementation strategies. The methodological approach used is suitable for the evaluation of complex implementation mechanisms. It has been proven to be an advantageous approach for formative evaluation.Trial registrationNCT03430336. First registration: 12/02/2018. ClinicalTrials.gov.

Parenting coordination – outcome measurement and prediction: Implications for research and practice

AbstractThis paper presents a formative evaluation study of the parenting coordination pilot program launched in Israel in 2016 at the initiative of the Ministry of Welfare and Social Affairs (At the time the program was launched, the Ministry of Labor, Social Affairs and Social Services) and JDC‐Ashalim. Data was collected on 131 participating families (262 parents) through self‐administered questionnaires filled out by the parents prior to the parenting coordination process (N = 204), telephone interviews with the parents following the process (N = 142), and reports by the parenting coordinators (PCs), submitted online at the conclusion of the process with each participating family (N = 131). Based on the logic model developed as a blueprint for the pilot program, the characteristics and needs of parents in high conflict were identified and desired outcome indicators were accordingly specified; quantitative and qualitative research tools were developed for the study, and the efficacy and outcomes of the parenting coordination pilot program were evaluated. Overall, the findings indicate a significant improvement in the desired outcome indicators. The implications of the findings for research and practice are discussed in the conclusion.

LifeLab: Co-Design of an Interactive Health Literacy Intervention for Socioeconomically Disadvantaged Adolescents'.

Low socioeconomic populations, when compared to more affluent groups, are at greater risk of initiating risky behaviours and consequently developing health complications. Health literacy has been identified as a possible means to improve and sustain positive health behaviours, with adolescence being a time point when such behaviours can be embedded. To develop a meaningful health intervention, it has been recommended that relevant stakeholders be included in the design phase. This formative evaluation study was the second phase of co-design of an engaging health literacy intervention ‘LifeLab’ with, and for, socioeconomically disadvantaged adolescents in Ireland. In Spring 2021, a series of co-design workshops (n = 17) were facilitated with a convenience sample of adolescents from socially disadvantaged areas (n = 22) to gather their perceptions, feedback, and suggested changes on the LifeLab learning activities that had emerged from Phase 1 of the work. The data was analysed using reflexive thematic analysis, resulting in the development of three themes: (i) preferred learning engagement strategies, (ii) practical and logistical considerations and (iii) ideas for LifeLab content. The results highlight the value in adopting a participatory approach, as participants offered an array of suggestions and details to maximise the potential for LifeLab to be contextually relevant and engaging; suggestions which will directly inform the development and implementation of the intervention.

Training the Next Generation of Diabetes Researchers: Evaluation of the Diabetes Action Canada Training and Mentoring Program

BackgroundThe Diabetes Action Canada Training and Mentoring (DAC-TM) Program launched in June 2017, with the goal of building capacity in the next generation of diabetes researchers in Canada in patient-oriented research (POR). MethodsWe conducted a program evaluation of the DAC-TM program using a sequential, mixed-methods research design. ResultsOur analysis of 82 surveys and 22 in-depth interviews from a wide range of DAC-TM Program stakeholders revealed consistent patterns in experience with the program. The training sessions were perceived to be well-organized, convenient educational opportunities to gain new knowledge about POR and become integrated into a community of practice of POR researchers who study diabetes and its complications in Canada. The content of the training was perceived to be useful and relevant to participants, although improvements could be made to help address the training needs of the broader DAC community. There is broad support for and appreciation of the mentorship awards, which were perceived to be appropriately targeted to early-career investigators. The mentor–mentee relationships were perceived to be positive, productive and career-advancing overall, but could benefit from a more strategic design and promote better connectivity to foster mentor–mentee relationships. In addition, feedback about opportunities to network and forge new connections was mixed and represents another opportunity for improvement to strengthen capacity building. ConclusionsFindings from this formative evaluation study show key strengths and opportunities to improve the DAC-TM Program, which can be used to help enhance its function and promote its long-term sustainability.

A Smartphone App for Supporting the Self-management of Daytime Urinary Incontinence in Adolescents: Development and Formative Evaluation Study of URApp.

BackgroundDaytime urinary incontinence (UI) is common in childhood and often persists into adolescence. UI in adolescence is associated with a range of adverse outcomes, including depressive symptoms, peer victimization, poor self-image, and problems with peer relationships. The first-line conservative treatment for UI is bladder training (standard urotherapy) that aims to establish a regular fluid intake and a timed schedule for toilet visits. The success of bladder training is strongly dependent on good concordance, which can be challenging for young people.ObjectiveThis paper aims to describe the development of a smartphone app (URApp) that aims to improve concordance with bladder training in young people aged 11 to 19 years.MethodsURApp was designed by using participatory co-design methods and was guided by the person-based approach to intervention design. The core app functions were based on clinical guidance and included setting a daily drinking goal that records fluid intake and toilet visits, setting reminders to drink fluids and go to the toilet, and recording progress toward drinking goals. The development of URApp comprised the following four stages: a review of current smartphone apps for UI, participatory co-design workshops with young people with UI for gathering user requirements and developing wireframes, the development of a URApp prototype, and the user testing of the prototype through qualitative interviews with 23 young people with UI or urgency aged 10 to 19 years and 8 clinicians. The app functions and additional functionalities for supporting concordance and behavior change were iteratively optimized throughout the app development process.ResultsYoung people who tested URApp judged it to be a helpful way of supporting their concordance with a timed schedule for toilet visits and drinking. They reported high levels of acceptability and engagement. Preliminary findings indicated that some young people experienced improvements in their bladder symptoms, including a reduction in UI. Clinicians reported that URApp was clinically appropriate and aligned with the best practice guidelines for bladder training. URApp was deemed age appropriate, with all clinicians reporting that they would use it within their own clinics. Clinicians felt URApp would be of particular benefit to patients whose symptoms were not improving or those who were not engaging with their treatment plans.ConclusionsThe next stage is to evaluate URApp in a range of settings, including pediatric continence clinics, primary care, and schools. This research is needed to test whether URApp is an effective (and cost-effective) solution for improving concordance with bladder training, reducing bladder symptoms, and improving the quality of life.

A Digital Platform for Facilitating Personalized Dementia Care in Nursing Homes: Formative Evaluation Study.

BackgroundCare personalization is key to the well-being of people with dementia according to person-centered care. With the development of the internet of things, a large quantity of personal data can be collected securely and reliably, which has the potential to facilitate care personalization for people with dementia. Yet, there are limited assistive technologies developed for this purpose, and the user acceptance of assistive technologies is low in nursing homes. Therefore, through a data-enabled design approach, a digital platform was developed for helping the care team in a nursing home to personalize dementia care, specifically in the management of behavioral and psychological dementia symptoms.ObjectiveThis study aimed to evaluate the digital platform in a real-life context with potential users from the following two aspects: (1) to explore if the digital platform could help with generating insights on the current state of each person with dementia and (2) to gather feedback on the digital platform from the care team.MethodsThe digital platform was deployed in the nursing home for 7 weeks and the data collected were visualized and presented to the care team via the digital platform. The visualizations were analyzed by the researchers for pattern detection. Meanwhile, the care team was asked to examine the visualizations and were interviewed for the following: (1) if any insights and actions were generated from the examination, (2) the usefulness of the digital platform, and (3) the improvements they would like to see.ResultsThe data collected on the digital platform demonstrated its potential for pattern detection. Insights were generated by the care team and categorized into “client level,” “ward level,” and “team level.” The corresponding actions taken by the care team were classified into “investigation” and “implementation.” User acceptance varied across the care team, and three aspects of improvement for the digital platform were identified.ConclusionsBy evaluating the digital platform, this study gained insights on applying data-enabled design for personalizing dementia care; besides, it offers future researchers some recommendations on how to integrate assistive technologies in the nursing home context.

Bispectral Index Alterations and Associations With Autonomic Changes During Hypnosis in Trauma Center Researchers: Formative Evaluation Study.

BackgroundPrevious work performed by our group demonstrated that intermittent reductions in bispectral index (BIS) values were found during neurofeedback following mindfulness instructions. Hypnosis was induced to enhance reductions in BIS values.ObjectiveThis study aims to assess physiologic relaxation and explore its associations with BIS values using autonomic monitoring.MethodsEach session consisted of reading a 4-minute baseline neutral script and playing an 18-minute hypnosis tape to 3 researchers involved in the BIS neurofeedback study. In addition to BIS monitoring, autonomic monitoring was performed, and this included measures of electromyography (EMG), skin temperature, skin conductance, respiratory rate, expired carbon dioxide, and heart rate variability. The resulting data were analyzed using two-tailed t tests, correlation analyses, and multivariate linear regression analyses.ResultsWe found that hypnosis was associated with reductions in BIS (P<.001), EMG (P<.001), respiratory rate (P<.001), skin conductance (P=.006), and very low frequency power (P=.04); it was also associated with increases in expired carbon dioxide (P<.001), skin temperature (P=.04), high frequency power (P<.001), and successive heart interbeat interval difference (P=.04) values. Decreased BIS values were associated with reduced EMG measures (R=0.76; P<.001), respiratory rate (R=0.35; P=.004), skin conductance (R=0.57; P<.001), and low frequency power (R=0.32; P=.01) and with increased high frequency power (R=−0.53; P<.001), successive heart interbeat interval difference (R=−0.32; P=.009), and heart interbeat interval SD (R=−0.26; P=.04) values.ConclusionsHypnosis appeared to induce mental and physical relaxation, enhance parasympathetic neural activation, and attenuate sympathetic nervous system activity, changes that were associated with BIS values. Findings from this preliminary formative evaluation suggest that the current hypnosis model may be useful for assessing autonomic physiological associations with changes in BIS values, thus motivating us to proceed with a larger investigation in trauma center nurses and physicians.

Neural network application to petrophysical and lithofacies analysis based on multi-scale data: An integrated study using conventional well log, core and borehole image data

Application of artificial neural network (ANN, e.g., Multi-layer perceptron, MLP) became widespread in the petroleum industry, especially in formation evaluation, reservoir characterization and modeling studies. In this study, the MLP technique is applied to estimate the lithofacies and petrophysical parameters including permeability and rock typing for the upper unit of Abu Roash G reservoir (ARG), Azhar field in Beni Suef basin. Major steps of the workflow started with lithofacies analysis using core and borehole image data to identify five lithofacies as follows; bioturbated mudstone, lenticular-bedded siltstone, massive laminated sandstone, flaser bedded sandstone, and faintly laminated sandstone, which were deposited in tidal conditions. A fully connected neural network was trained and tested on data for lithofacies prediction in wells with no core data and borehole image. Core permeability was estimated by using different variables including effective porosity, bulk density, volume of shale and volume of sand, it was found that neural network prediction for permeability was very satisfactory and can be applied on wider scale in other uncored wells. Rock typing was constructed in cored wells using several methods including porosity-permeability plot integrated with the Winland R35, flow zone indicator-reservoir quality index (FZI-RQI). Neural network classification method was trained and tested on data for rock type estimation in the uncored zones based on calculated petrophysical log data including permeability, flow zone indicator, reservoir quality index and effective porosity. To get more accurate predicted data, the log data was calibrated firstly using porosity and permeability core data.

Using a Personal Health Library-Enabled mHealth Recommender System for Self-Management of Diabetes Among Underserved Populations: Use Case for Knowledge Graphs and Linked Data.

BackgroundTraditionally, digital health data management has been based on electronic health record (EHR) systems and has been handled primarily by centralized health providers. New mechanisms are needed to give patients more control over their digital health data. Personal health libraries (PHLs) provide a single point of secure access to patients' digital health data and enable the integration of knowledge stored in their digital health profiles with other sources of global knowledge. PHLs can help empower caregivers and health care providers to make informed decisions about patients’ health by understanding medical events in the context of their lives.ObjectiveThis paper reports the implementation of a mobile health digital intervention that incorporates both digital health data stored in patients’ PHLs and other sources of contextual knowledge to deliver tailored recommendations for improving self-care behaviors in diabetic adults.MethodsWe conducted a thematic assessment of patient functional and nonfunctional requirements that are missing from current EHRs based on evidence from the literature. We used the results to identify the technologies needed to address those requirements. We describe the technological infrastructures used to construct, manage, and integrate the types of knowledge stored in the PHL. We leverage the Social Linked Data (Solid) platform to design a fully decentralized and privacy-aware platform that supports interoperability and care integration. We provided an initial prototype design of a PHL and drafted a use case scenario that involves four actors to demonstrate how the proposed prototype can be used to address user requirements, including the construction and management of the PHL and its utilization for developing a mobile app that queries the knowledge stored and integrated into the PHL in a private and fully decentralized manner to provide better recommendations.ResultsTo showcase the main features of the mobile health app and the PHL, we mapped those features onto a framework comprising the user requirements identified in a use case scenario that features a preventive intervention from the diabetes self-management domain. Ongoing development of the app requires a formative evaluation study and a clinical trial to assess the impact of the digital intervention on patient-users. We provide synopses of both study protocols.ConclusionsThe proposed PHL helps patients and their caregivers take a central role in making decisions regarding their health and equips their health care providers with informatics tools that support the collection and interpretation of the collected knowledge. By exposing the PHL functionality as an open service, we foster the development of third-party applications or services and provide motivational technological support in several projects crossing different domains of interest.

Learning to Use Narrative Function Words for the Organization and Communication of Experience

How do people learn to talk about the causal and temporal relations between events, and the motivation behind why people do what they do? The narrative practice hypothesis of Hutto and Gallagher holds that children are exposed to narratives that provide training for understanding and expressing reasons for why people behave as they do. In this context, we have recently developed a model of narrative processing where a structured model of the developing situation (the situation model) is built up from experienced events, and enriched by sentences in a narrative that describe event meanings. The main interest is to develop a proof of concept for how narrative can be used to structure, organize and describe experience. Narrative sentences describe events, and they also define temporal and causal relations between events. These relations are specified by a class of narrative function words, including “because, before, after, first, finally.” The current research develops a proof of concept that by observing how people describe social events, a developmental robotic system can begin to acquire early knowledge of how to explain the reasons for events. We collect data from naïve subjects who use narrative function words to describe simple scenes of human-robot interaction, and then employ algorithms for extracting the statistical structure of how narrative function words link events in the situation model. By using these statistical regularities, the robot can thus learn from human experience about how to properly employ in question-answering dialogues with the human, and in generating canonical narratives for new experiences. The behavior of the system is demonstrated over several behavioral interactions, and associated narrative interaction sessions, while a more formal extended evaluation and user study will be the subject of future research. Clearly this is far removed from the power of the full blown narrative practice capability, but it provides a first step in the development of an experimental infrastructure for the study of socially situated narrative practice in human-robot interaction.

Formative Evaluation of a Home-Based Physical Activity Intervention for Adolescent Girls-The HERizon Project: A Randomised Controlled Trial.

Background. This is a formative evaluation study of the HERizon Project, a home-based multi-component physical activity (PA) intervention for adolescent girls in the UK and Ireland. Although not intended, this study coincided with the initial COVID-19 lockdown restrictions. Methods. A total of 42 female participants, aged 13 to 16 years old (mean = 14.2, SD = 1.1), were randomly allocated to: (i) the HERizon group (n = 22) or (ii) the wait-list control group (n = 20). Participants in the six-week HERizon group were asked to complete three PA sessions each week and engage in weekly behaviour change support video calls. The primary outcome measure was self-reported habitual PA. Secondary outcomes measures included cardiorespiratory fitness (20 m shuttle run), muscular strength (standing long jump), muscular endurance (push up test), and psychosocial outcomes (Perceived Competence Scale, Body Appreciation Scale, Self-Esteem Questionnaire, Behavioural Regulation in Exercise Questionnaire). Quantitative and qualitative process evaluation data were also collected. Outcome measures were assessed at baseline and after the six-week intervention. Results. There was no significant change in habitual PA between groups (LMM group*time interaction: p = 0.767). The HERizon group had significantly increased cardiorespiratory fitness (p = 0.001), muscular endurance (p = 0.022), intrinsic motivation (p = 0.037), and body appreciation (p < 0.003) in comparison to the wait-list control group. All participants in the intervention group completed the intervention and compliance to the intervention was high (participants completed 18 ± 2 sessions). Conclusions. Although no change in PA was observed, HERizon resulted in improved physical fitness and psychosocial outcomes. These preliminary findings, alongside positive findings for feasibility and acceptability, highlight potential benefits from the home-based intervention, thus further investigation is warranted.

Implementing Competency-Based Medical Education in Family Medicine: A Narrative Review of Current Trends in Assessment.

The implementation of effective competency-based medical education (CBME) relies on building a coherent and integrated system of assessment across the continuum of training to practice. As such, the developmental progression of competencies must be assessed at all stages of the learning process, including continuing professional development (CPD). Yet, much of the recent discussion revolves mostly around residency programs. The purpose of this review is to synthesize the findings of studies spanning the last 2 decades that examined competency-based assessment methods used in family medicine residency and CPD, and to identify gaps in their current practices. We adopted a modified form of narrative review and searched five online databases and the gray literature for articles published between 2000 and 2020. Data analysis involved mixed methods including quantitative frequency analysis and qualitative thematic analysis. Thirty-seven studies met inclusion criteria. Fourteen were formal evaluation studies that focused on the outcome and impact evaluation of assessment methods. Articles that focused on formative assessment were prevalent. The most common levels of educational outcomes were performance and competence. There were few studies on CBME assessment among practicing family physicians. Thematic analysis of the literature identified several challenges the family medicine educational community faces with CBME assessment. We recommend that those involved in health education systematically evaluate and publish their CBME activities, including assessment-related content and evaluations. The highlighted themes may offer insights into ways in which current CBME assessment practices might be improved to align with efforts to improve health care.

Achieving Better Educational Practices Through Research Evidence: A Critical Analysis and Case Illustration of Benefits and Challenges

Purpose: We examine considerations regarding the positive contributions of evidence accountability and challenges that frustrate educators in gaining access to the needed product information. Design/Approach/Methods: We review the research literature on the multiple characteristics of evidence relative to consumer (practitioner) interests. We then examine, through a “case illustration” of an initiative in a large school district, a second challenge for evidence usage—conducting viable studies and interpreting outcomes from comprehensive interventions in complex educational systems. Findings: Despite attention being given to rigorous evidence, consumers report preferring peer recommendations and local pilot studies as sources. In our case illustration, we found that the availability of evidence from comprehensive formative evaluation studies was viewed by stakeholders as positively contributing to program implementation quality and sustainability over time. Originality/Value: We use a real-world “case illustration” of a complex initiative in a large, diverse school district to illustrate how current policies and expectations regarding evidence support for educational programs is filtered through multiple agendas and personal needs of key stakeholders. Consequently, evaluators acquire nontraditional roles that go beyond routine execution of rigorous studies. Given these factors, we offer recommendations for fostering more meaningful and objective interpretations and usage of evidence by local stakeholders.