Inform Tool Development Research Articles

Adults perceive similar ultra-processed and minimally processed foods as having different levels of healthfulness: Development of stimuli for an ultra-processed food-related implicit association task.

Background: Ultra-processed food (UPF) consumption is excessive across the United States (US). Understanding perceptions of food based on processing is critical to elucidate reasons for UPF intake and inform tool development for measuring related cognition. Aim: Design and evaluate perceptions of UPF and non-UPF food among general US adults. Hypothesis: Significant differences exist in perceived healthfulness across similar UPF and non-UPF foods. Methods: Photographs of foods were taken to represent eight pairs of similar UPFs and non-UPFs and presented in a Qualtrics survey with questions to gauge healthfulness from 0 to 10 (0 = completely unhealthy, 10 = completely healthy). Paired-samples sign tests were used to detect differences in ratings within pairs. Results: Participants (n = 100) rated stimuli within all pairs significantly differently (P < 0.001), indicating different perceptions of UPF and non-UPF. Conclusion: Adults perceive similar UPF and non-UPF as having different levels of healthfulness. Reasons for these differences warrant investigation.

Development of a Tool to Assess Surgical Resident Competence On-Call: The Western University Call Assessment Tool (WUCAT)

A central tenet of competency-based medical education is the formative assessment of trainees. There are currently no assessments designed to examine resident competence on-call, despite the on-call period being a significant component of residency, characterized by less direct supervision compared to daytime. The purpose of this study was to design a formative on-call assessment tool and collect valid evidence on its application. Nominal group technique was used to identify critical elements of surgical resident competence on-call to inform tool development. The tool was piloted over six months in the Division of Plastic & Reconstructive Surgery at our institution. Quantitative and qualitative evidence was collected to examine tool validity. A ten-item tool was developed based on the consensus group results. Sixty-three assessments were completed by seven staff members on ten residents during the pilot. The tool had a reliability coefficient of 0.67 based on a generalizability study and internal item consistency was 0.92. Scores were significantly associated with years of training. We found the tool improved the quantity and structure of feedback given and that the tool was considered feasible and acceptable by both residents and staff members. The Western University Call Assessment Tool (WUCAT) has multiple sources of evidence supporting its use in assessing resident competence on-call.

505 Use of Implementation Science to Identify Implementation Determinants of Chronic Obstructive Pulmonary Disease Practice Guidelines

OBJECTIVES/GOALS: COPD is a progressive airways disease that results in death or disability. There is poor uptake of clinical guidelines (CPG) to manage COPD and studies to bridge this implementation gap have shown inconsistent results. Using implementation science principles we aim to understand COPD-CPG implementation determinants from providers’perspective. METHODS/STUDY POPULATION: The study is being conducted in ten VA Primary Care Clinics. Guided by the Consolidated Framework for Implementation Research (CFIR), a conceptual framework developed to guide systematic assessment of multilevel implementation contexts, we are using semi-structured guides to conduct key informant qualitative interviews (physicians, physician extenders and nurses), to support a formative evaluation. CFIR domains relevant to the study were determined by a multidisciplinary team. Informants are identified through online outreach and voluntary participation. Sampling adequacy will be assessed by achievement of code saturation. A qualitative template analysis will be used to summarize the barriers and facilitators of each component of COPD-CPG organized by CFIR-domain. RESULTS/ANTICIPATED RESULTS: We anticipate a list of modifiable and non-modifiable contextual, recipient (provider and patient), and COPD CPG content (innovation) barriers to implementation. Many settings do not have critical elements of these CPG, such as a standardized inhaler education/assessment pathway, patient education material, or pulmonary rehabilitation referral pathway. Existing literature indicate reasons behind the insufficient uptake of COPD CPG include low familiarity with guidelines, perception of minimal value of guidelines by physicians, and time constraints; we will present contextual, recipient and innovation determinants specific to our setting. DISCUSSION/SIGNIFICANCE: This comprehensive assessment of barriers and facilitators to COPD-CPG will inform tool development and implementation strategies identification to improve COPD CPG uptake. COPD is the most common veteran lung disease. Improvement in COPD care has enormous potential for benefit for local veterans, as well as potential for wider dissemination.

Assessing Polyvictimization in a Family Justice Center: Lessons Learned From a Demonstration Project

There is a growing interest in developing comprehensive assessments that measure intimate partner violence (IPV) alongside other adverse events that correlate with IPV and compound its effects. One promising line of research in this area has focused on the impact of exposure to multiple types of victimization, i.e., polyvictimization. The purpose of this study is to examine the experience of administration of a polyvictimization tool from staff and client perspectives in order to inform future tool developments and assessment procedures. Qualitative interviews and focus groups with clients and staff from a family justice center who had experience with the assessment tool were used to identify strengths and challenges of the assessment too and inform future tool development. Findings demonstrate that an assessment tool provides the space for clients to talk about trauma and facilitate empowerment, while providing the opportunity for psychoeducation and service referrals. Concerns about the assessment tool included adverse reactions without proper framing and language, as well as shifting the emphasis from screening for adversities toward strengths, coping skills, and resilience. Implications for future measurement development and establishing best practices in polyvictimization assessment are discussed, with an emphasis on the benefits of social service agencies utilizing assessment tools.

Exploring the Experiences and Information Needs of Parents Caring for a Child With a Urinary Tract Infection: A Qualitative Study.

Urinary tract infections (UTIs) are common in children, however, clinical practice variations can leave parents unsure how to care for their child. We aim to develop knowledge tools that provide evidence-based information about pediatric UTIs. To inform tool development, we asked parents to share their experiences and information needs in caring for a child with a UTI. Using qualitative description methods, 16 semistructured interviews were conducted with 18 parents. Parents were recruited through the emergency department (ED) of a major Canadian urban pediatric hospital. Five major themes emerged: (1) parent descriptions of their child’s symptoms and behaviors; (2) UTIs have an effect on the entire family; (3) reasons for going to the ED; (4) parent experiences with UTI treatment, management, and follow-up strategies for their child; and (5) parent information needs and preferred information sources for UTIs. Our findings highlight the diversity of UTI symptoms children experience, which causes uncertainty and confusion for parents. This diversity suggests the development of knowledge tools for parents about UTIs is needed.

Measuring Learning and Engagement during Debriefing: A New Instrument

BackgroundDebriefing is considered to be where the bulk of learning takes place. Examination of debriefing is needed to help determine if facilitation methods effectively contribute to the learning process. More instruments are needed to evaluate all key debriefing elements, and no tools exist that evaluate participant engagement in the learning process. MethodKolb’s experiential learning theory was used to inform tool development. The focus of the new instrument is behaviors that indicate engagement with the learning process as the learner moves through Kolb’s learning cycle. A qualitative study of debriefing facilitators was performed, and a behaviorally anchored instrument was then created. A subject matter expert review process was used to help establish validity. Reliability was evaluated in a research study looking at inter-rater consistency. ResultsThe new instrument is used to count learning and engagement behaviors that are exhibited in a debriefing session. ConclusionThe instrument is ready for use in debriefing research studies, helping evaluate participant engagement in the learning process by identifying learning and engagement behaviors that they may exhibit during a debriefing.

Identifying and selecting implementation theories, models and frameworks: a qualitative study to inform the development of a decision support tool

BackgroundImplementation theories, models and frameworks offer guidance when implementing and sustaining healthcare evidence-based interventions. However, selection can be challenging given the myriad of potential options. We propose to inform a decision support tool to facilitate the appropriate selection of an implementation theory, model or framework in practice. To inform tool development, this study aimed to explore barriers and facilitators to identifying and selecting implementation theories, models and frameworks in research and practice, as well as end-user preferences for features and functions of the proposed tool.MethodsWe used an interpretive descriptive approach to conduct semi-structured interviews with implementation researchers and practitioners in Canada, the United States and Australia. Audio recordings were transcribed verbatim. Data were inductively coded by a single investigator with a subset of 20% coded independently by a second investigator and analyzed using thematic analysis.ResultsTwenty-four individuals participated in the study. Categories of barriers/facilitators, to inform tool development, included characteristics of the individual or team conducting implementation and characteristics of the implementation theory, model or framework. Major barriers to selection included inconsistent terminology, poor fit with the implementation context and limited knowledge about and training in existing theories, models and frameworks. Major facilitators to selection included the importance of clear and concise language and evidence that the theory, model or framework was applied in a relevant health setting or context. Participants were enthusiastic about the development of a decision support tool that is user-friendly, accessible and practical. Preferences for tool features included key questions about the implementation intervention or project (e.g., purpose, stage of implementation, intended target for change) and a comprehensive list of relevant theories, models and frameworks to choose from along with a glossary of terms and the contexts in which they were applied.ConclusionsAn easy to use decision support tool that addresses key barriers to selecting an implementation theory, model or framework in practice may be beneficial to individuals who facilitate implementation practice activities. Findings on end-user preferences for tool features and functions will inform tool development and design through a user-centered approach.

MyHEARTSMAP: development and evaluation of a psychosocial self-assessment tool, for and by youth

BackgroundPaediatric mental health-related visits to the emergency department are rising. However, few tools exist to identify concerns early and connect youth with appropriate mental healthcare. Our objective was to develop...

An interactive decision making tool to improve continuity of carer in maternity health service re-design

Background: At a statewide forum in November 2016 it was identified that maternity services in Queensland improve woman centred care by strengthening four key maternity areas of service delivery: Building a Collaborative Leadership Culture, Identification and Management of Risk in Pregnancy, Models of Care and Workforce; and Aboriginal and Torres Strait Islander Peoples. Four maternity service action groups (MSAG) were tasked to address these key areas. Aim: Develop an interactive decision making framework (DMF) tool to increase maternity continuity of carer models. Innovation description: An external provider to Queensland Health was procured to build the interactive electronic tool to support health services plan, develop and transition to contemporary evidenced based models. The DMF supports services to contextualise delivery of continuity of carer to local circumstances. Considerations included service demand, geographical circumstances, required resources, and optimal maternity bio psychosocial outcomes. Steps undertaken to inform tool development included a literature review, maternity site visits and a forum canvassing the collective vision of service leaders, clinicians and consumers. The essential components for the interactive tool were refined between all stakeholders. Products developed by the four MSAGs were also integrated to the DMF as a library of complementary resources to further inform health services during their redesign process, and when using the tool. Evaluation: An exemplar site where recent service redesign occurred was chosen to test efficacy of the tool. Modifications of the tool are being made before testing with rural, regional and metropolitan sites. Early adopter sites have been identified to implement the DMF tool in March 2018. These outcomes will be presented at conference. Generalisability: One third of Queensland Hospital and Health Services will have tested the tool providing confidence for use in Queensland. The tool has applicability to other similar jurisdictions looking to expand continuity of carer models.

Evaluation and refinement of a handheld health information technology tool to support the timely update of bedside visual cues to prevent falls in hospitals.

To evaluate clinicians' perspectives, before and after clinical implementation (i.e. trial) of a handheld health information technology (HIT) tool, incorporating an iPad device and automatically generated visual cues for bedside display, for falls risk assessment and prevention in hospital. This pilot study utilized mixed-methods research with focus group discussions and Likert-scale surveys to elicit clinicians' attitudes. The study was conducted across three phases within two medical wards of the Queen Elizabeth Hospital. Phase 1 (pretrial) involved focus group discussion (five staff) and surveys (48 staff) to elicit preliminary perspectives on tool use, benefits and barriers to use and recommendations for improvement. Phase 2 (tool trial) involved HIT tool implementation on two hospital wards over consecutive 12-week periods. Phase 3 (post-trial) involved focus group discussion (five staff) and surveys (29 staff) following tool implementation, with similar themes as in Phase 1. Qualitative data were evaluated using content analysis, and quantitative data using descriptive statistics and logistic regression analysis, with subgroup analyses on user status (P ≤ 0.05). Four findings emerged on clinicians' experience, positive perceptions, negative perceptions and recommendations for improvement of the tool. Pretrial, clinicians were familiar with using visual cues in hospital falls prevention. They identified potential benefits of the HIT tool in obtaining timely, useful falls risk assessment to improve patient care. During the trial, the wards differed in methods of tool implementation, resulting in lower uptake by clinicians on the subacute ward. Post-trial, clinicians remained supportive for incorporating the tool into clinical practice; however, there were issues with usability and lack of time for tool use. Staff who had not used the tool had less appreciation for it improving their understanding of patients' falls risk factors (odds ratio 0.12), or effectively preventing hospital falls (odds ratio 0.12). Clinicians' recommendations resulted in subsequent technological refinement of the tool, and provision of an additional iPad device for more efficient use. This study adds to the limited pool of knowledge about clinicians' attitudes toward health technology use in falls avoidance. Clinicians were willing to use the HIT tool, and their concerns about its usability were addressed in ongoing tool improvement. Including end-users in the development and refinement processes, as well as having high staff uptake of new technologies, is important in improving their acceptance and usage, and in maximizing beneficial feedback to further inform tool development.