Continuing Professional Development Interventions Research Articles

Learning Outcomes and Educational Effectiveness of Social Media as a Continuing Professional Development Intervention for Practicing Surgeons: A Systematic Review and Narrative Synthesis.

The objective of this systematic review was to assess the learning outcomes and educational effectiveness of social media as a continuing professional development intervention for surgeons in practice. Social media has the potential to improve global access to educational resources and collaborative networking. However, the learning outcomes and educational effectiveness of social media as a continuing professional development (CPD) intervention are yet to be summarized. We searched MEDLINE and Embase databases from 1946 to 2022. We included studies that assessed the learning outcomes and educational effectiveness of social media as a CPD intervention for practicing surgeons. We excluded studies that were not original research, involved only trainees, did not evaluate educational effectiveness, or involved an in-person component. The 18-point Medical Education Research Study Quality Instrument (MERSQI) was used for quality appraisal. Learning outcomes were categorized according to Moore's Expanded Outcomes Framework (MEOF). A total of 830 unique studies revealed 14 studies for inclusion. The mean MERSQI score of the included studies was 9.0 ± 0.8. In total, 3227 surgeons from 105 countries and various surgical specialties were included. Twelve studies (86%) evaluated surgeons' satisfaction (MEOF level 2), 3 studies (21%) evaluated changes in self-reported declarative or procedural knowledge (MEOF levels 3A and 3B), 1 study (7%) evaluated changes in self-reported competence (MEOF level 4), and 5 studies (36%) evaluated changes in self-reported performance in practice (MEOF level 5). No studies evaluated changes in patient or community health (MEOF levels 6 and 7). The use of social media as a CPD intervention among practicing surgeons is associated with improved self-reported declarative and procedural knowledge, self-reported competence, and self-reported performance in practice. Further research is required to assess whether social media use for CPD in surgeons is associated with improvements in higher level and objectively measured learning outcomes.

Learning Outcomes and Educational Effectiveness of E-Learning as a Continuing Professional Development Intervention for Practicing Surgeons and Proceduralists: A Systematic Review

Electronic learning (e-Learning) has become a prevalent mode of delivering medical education. We aimed to determine the learning outcomes and educational effectiveness of e-Learning as a continuing professional development (CPD) intervention for practicing surgeons and proceduralists. We searched MEDLINE databases and included studies reporting learning outcomes of e-learning CPD interventions for practicing surgeons and physicians performing technical procedures. We excluded articles only studying surgical trainees and those not reporting learning outcomes. Two reviewers independently screened, extracted data, and assessed study quality using the Critical Appraisal Skills Programme (CASP) tools. Learning outcomes and educational effectiveness were categorized using Moore's Outcomes Framework (PROSPERO: CRD42022333523). Of 1307 identified articles, 12 were included- 9 cohort studies, one randomized controlled trial and 2 qualitative studies, with a total of 2158 participants. Eight studies were rated as moderate, five as strong, and 2 as weak in study quality. E-Learning CPD interventions included web-based modules, image recognition, videos, a repository of videos and schematics, and an online journal club. Seven studies reported participants' satisfaction with the e-Learning interventions (Moore's Level 2), 4 reported improvements in participants' declarative knowledge (Level 3a), 1 reported improvements in procedural knowledge (Level 3b) and five reported improvements in participants' procedural competence in an educational setting (Level 4). No studies demonstrated improvements in participants' workplace-based performance, the health of patients, or community health (Levels 5-7). E-Learning as a CPD educational intervention is associated with high satisfaction and improvements in knowledge and procedural competencies of practicing surgeons and proceduralists in an educational setting. Future research is required to investigate whether e-Learning is associated with higher-level learning outcomes.

Paving the Way Forward for Evidence-Based Continuing Professional Development.

Continuing professional development (CPD) fosters lifelong learning and enables health care providers to keep their knowledge and skills current with rapidly evolving health care practices. Instructional methods promoting critical thinking and decision making contribute to effective CPD interventions. The delivery methods influence the uptake of content and the resulting changes in knowledge, skills, attitudes, and behavior. Educational approaches are needed to ensure that CPD meets the changing needs of health care providers. This article examines the development approach and key recommendations embedded in a CE Educator's toolkit created to evolve CPD practice and foster a learning experience that promotes self-awareness, self-reflection, competency, and behavioral change. The Knowledge-to-Action framework was used in designing the toolkit. The toolkit highlighted three intervention formats: facilitation of small group learning, case-based learning, and reflective learning. Strategies and guidelines to promote active learning principles in CPD activities within different modalities and learning contexts were included. The goal of the toolkit is to assist CPD providers to design educational activities that optimally support health care providers' self-reflection and knowledge translation into their clinical environment and contribute to practice improvement, thus achieving the outcomes of the quintuple aim.

Barriers and Facilitators to the Implementation and Adoption of a Continuous Quality Improvement Program in Surgery: A Case Study.

As postoperative adverse events (AEs) drive worsened patient experience, longer length of stay, and increased costs of care, surgeons have long sought to engage in innovative approaches aimed at reducing AEs to improve the quality and safety of surgical care. While data-driven AE performance measurement and feedback (PMF) as a form of continuing professional development (CPD) has been presented as a possible approach to continuous quality improvement (CQI), little is known about the barriers and facilitators that influence surgeons' engagement and uptake of these CPD programs. The purpose of this knowledge translation informed CPD study was to examine surgeons' perspectives of the challenges and facilitators to participating in surgical CQI with the broader objective of enhancing future improvements of such CPD interventions. Using Everett Rogers diffusion of innovations framework as a sampling frame, the participants were recruited across five surgical divisions. An exploratory case study approach, including in-depth semistructured interviews, was employed. Interview transcripts were analyzed and directly coded using the Theoretical Domains Framework. Directed coding yielded a total of 527 coded barriers and facilitators to behavior change pertaining to the implementation and adoption of PMF with the majority of barriers and facilitators coded in four key theoretical domains environmental context and resources, social influences, knowledge, and beliefs about consequences. A key barrier was the lack of support from the hospital necessitating surgeons' self-funding their own PMF programs. Facilitators included having a champion to drive CQI and using seminars to facilitate discussions around CQI principles and practices. This study identified multiple barriers and facilitators to surgeons' engagement and uptake of a data-driven PMF system in surgery. A key finding of the study was the identification of the influential role of positive deviance seminars as a quality improvement and patient safety mechanism that encourages surgeon engagement in PMF systems.

Using Administrative Data in Primary Care to Evaluate the Effectiveness of a Continuing Professional Development Program Focused on the Management of Patients Living With Obesity.

There are guidelines for referral to medical and/or surgical weight loss interventions (MSWLI) in Ontario; however, only about one-third of eligible patients in our region are being referred for consideration of MSWLI. A planning committee, including a registered dietician, psychiatrist, endocrinologist, bariatric surgeon, family physician, and educationalists, developed an interdisciplinary continuing professional development (CPD) program focused on practical approaches to the management of patients living with obesity. The Kirkpatrick model was used to evaluate the educational outcomes of the CPD program specifically focusing on Level-2, -3, and -4 outcomes based on self-reported questionnaire and health administrative data. Eighteen primary care providers from the CPD program agreed to participate in this study, and 16 primary care providers (89%) completed the postintervention questionnaire and granted us access to their MSWLI referral data; 94% of study participants reported changes to their knowledge, comfort, and confidence (Level 2), as well as expected change in their future behaviour (Level 3) following the CPD program. However, there was no change in Kirkpatrick Level-4 outcomes, despite more than 90% of participants indicating that they will be making changes to their practice after the program. The CPD program in our study was overwhelmingly well received and participants reported knowledge (Level 2) and behavioural (Level 3) changes following participation; however, there was no detectable change in their clinical practice (Level 4). The methodology described in our proof-of-concept study can be modified and adopted to evaluate Level-4 outcomes in other studies of effectiveness of CPD interventions.

Science of Learning Strategy Series: Article 3, Interleaving.

Interleaving is an evidence-based, learning-science strategy that is relevant to the planning and implementation of continuing professional development (CPD). Mixing related but different areas of study forces the brain to reconcile the relationship between the areas while understanding each area well. By doing so, interleaving increases the likelihood of mastery and memory. Research from cognitive psychology and neuroscience provides the rationale for interleaving, and examples of its implementation in health profession education have begun to appear in the literature. If utilized appropriately, some common CPD interventions can leverage interleaving. Through increased understanding, CPD participants can benefit from interleaving by making more-informed educational choices, and CPD planners can benefit in efforts to improve educational activities.

Exploring the Continuing Professional Development (CPD) Needs of Basic Schoolteachers in Ghana

This study aims to understand teachers’ perceptions of their professional development needs and examine how these needs are utilised to design CPD interventions for teachers in Ghana. The study uses a qualitative case study research approach to collect data from 25 teachers and five key informants in in-depth interviews. The results demonstrate teachers’ on-going learning needs in pedagogical content knowledge, technology integration in teaching and learning, and school and student management practices. Teachers also revealed that existing CPD interventions inadequately address their specific development needs. Teachers expressed their lack of voice and self-directedness in CPD program planning, design, and implementation, which they attributed as a significant cause of CPD ineffectiveness. Thus, the study recommends that teachers’ learning needs be formed as the pivot around which CPD is organised; to increase teacher participation and CPD program effectiveness.

Participant-reported effect of an Indigenous health continuing professional development initiative for specialists

BackgroundHealth outcomes of Indigenous patients are impacted by culturally unsafe specialty care environments. The ‘Educating for Equity (E4E)’ program is a continuing professional development (CPD) intervention which incorporates skill-based teaching to improve Indigenous patient experiences and outcomes in healthcare interactions.MethodsThe E4E program was delivered to rheumatologists in two phases, each delivered as experiential learning workshops where participants engaged with and applied course content within an interactive format focusing on real-time feedback. The phase 1 workshop focused on skill development of E4E Framework concepts and principles. Phase 2 concentrated on building capacity for teaching of E4E content. Evaluation of the program’s effectiveness was through longitudinal responses to the Social Cultural Confidence in Care Survey (SCCCS), self-reported strategies employed to address social issues and improve therapeutic relationships, engagement with teaching others, and satisfaction with the program.ResultsTwo cohorts of participants have participated in the program (n = 24 Phase 1, n = 10 Phase 2). For participants completing both phases of training, statistically significant improvements were observed in exploring social factors with patients, gaining knowledge and skills related to cultural aspects of care, improved communication and relationship building, and reflections on held stereotypes. Strategies to address social issues and build therapeutic relationships remained consistent throughout participation, while the training enhanced exploration and confidence to ask about cultural and traditional practices, and stronger communication strategies for exploring beliefs, expectations, social barriers, and residential school impacts on health. Participants reported feeling prepared to teach Indigenous health concepts to others and subsequently lead teaching with residents, fellows, and allied health professionals. Satisfaction with the delivery and content of the workshops was high, and participants valued interactions with peers in learning.ConclusionsThis CPD intervention had a beneficial impact on self-reported confidence and enhanced practice strategies to engage with Indigenous patients.

A Unique Model for Developing Continuing Education Resources for Health Professionals in Geographically Dispersed Health Organizations.

The provision of high quality, equitable, and accessible education resources is a key to supporting continuing professional development (CPD) in health organizations. The Health Education and Training Institute (HETI) and its operational model for districts (District HETI Operational Model), is a novel approach that supports this imperative for over 155,000 staff working across the state of New South Wales (NSW), Australia. The model uses three principles in education and training development. These include collaborative partnerships, rapid resource development, and effective leadership within virtual teams. A state-wide learning management system has been implemented to support this initiative. Over 451 standardized, education, and training online modules have been developed for medical, nursing and midwifery, dental, allied health, and nonclinical NSW Health employees since 2013. These educational resources are accessible 24 hours a day. Cost-effective online programs have enabled more than 13.6 million learner completions. Program evaluations highlight the value of these resources in providing a positive learning experience. Furthermore, the District HETI Operational Model has been recognized by multiple national and international excellence awards in learning and development. The principles of the District HETI Operational Model apply to other health organizations that may choose to adopt a similar model. Such a model may support equity of access to contemporary, standardized, evidence-based education resources for health professionals working across geographically and clinically diverse environments. Implementation of a similar model for future CPD interventions warrants consideration by practitioners, researchers, and policymakers.

Science of Learning Strategy Series: Article 1, Distributed Practice

Distributed practice is an evidence-based, learning-science strategy that is relevant to the planning and implementation of continuing professional development (CPD). Spacing-out study or practice over time allows the brain multiple opportunities to process new and complex information in an efficient way, thus increasing the likelihood of mastery and memory. Research from cognitive psychology and neuroscience provide the rationale for distributed practice, and examples of its implementation in health professions education have begun to appear in the literature. If used appropriately or extended creatively, some common CPD interventions can fully leverage distributed practice. Through increased understanding, CPD planners can benefit from distributed practice in efforts to improve educational activities, and CPD participants can benefit by making more informed educational choices.

Evaluating the role of video in supporting reflection beyond INSET

This article considers the impact of a video-based initiative aimed at sustaining engagement, reflection and impact beyond a three-week INSET (in-service teacher education and training) programme. Teachers worked with mentors on a one to one basis in a process of reflecting on their own videos and clips from other teachers. The study uses interviews, recordings and transcripts of video-based talk to evaluate the design of this video-led mentor CPD (continuing professional development) intervention and to evaluate the nature of video-based reflection in this process. The analysis focused on four themes that emerged as significant in the analysis of recordings and project documents. Not surprisingly we found that reflection was orientated to INSET (‘bootcamp’) content although videos were sometimes sources of resistance to this content. Interactions varied based on a number of factors related to the mentor role and video type. Our findings confirmed previous studies regarding video length and type and relationship to reflection. We also found that language choice (Thai/English) was significant and that teachers may prefer to reflect in L1. We provide suggestions for how a video-based CDP intervention might help to promote reflection in teachers as well as recommendations for other educators considering video-based alternatives.

A multifaceted continuing professional development intervention to move stroke rehabilitation guidelines into professional practice: A feasibility study

ABSTRACT Introduction Rehabilitation post-stroke is critical for maximizing patient outcomes. This study assessed the feasibility of implementing and evaluating a continuing professional development (CPD) intervention aimed at increasing the uptake of stroke best practice guidelines among physiotherapists (PTs), occupational therapists (OTs) and speech-language pathologists (SLPs) in six university-affiliated stroke rehabilitation centers in Quebec, Canada. Method Twelve stroke best practice recommendations with reflective tools were sent weekly by e-mail. Participants’ eligibility criteria included: a) profession; b) practicing more than 1 year in a stroke rehabilitation program; c) fluency in French or English; and d) basic computer literacy. Feasibility (operationalized via participation, satisfaction and relevance), cognitive impact, perceived application in practice and expected patient outcomes were measured over 24 weeks using three questionnaires and analyzed using descriptive statistics. Results The sample totaled 62 of 133 eligible (47%) clinicians. Satisfaction, relevance and cognitive impact of delivered information varied across disciplines and recommendations. Agreement with the recommendations was high across disciplines. On average, three-interdisciplinary recommendations (related to post-stoke depression, post-stoke fatigue and patients’ and caregivers’ learning needs) were rated as the most relevant for at least one patient. The majority of clinicians would use the recommendations for a specific patient and expected health benefits by applying those recommendations. Conclusion This study demonstrated the feasibility of assessing the impact of a CPD intervention in stroke rehabilitation uptake and informed the design of a research program aimed at increasing the use of stroke evidence-based rehabilitation interventions.

Behavior Change Techniques in Continuing Professional Development.

Continuing professional development (CPD) is a widely used and evolving set of complex interventions that seeks to update and improve the knowledge, skills, and performance of health care professionals to ultimately improve patient care and outcomes. While synthesized evidence shows CPD in general to be effective, effects vary, in part due to variation in CPD interventions and limited understanding of CPD mechanisms of action. We introduce two behavioral science tools-the Behavior Change Technique Taxonomy version 1 and the Theoretical Domains Framework-that can be used to characterize the content of CPD interventions and the determinants of behaviour potentially targeted by the interventions, respectively. We provide a worked example of the use of these tools in coding the educational content of 43 diabetes quality improvement trials containing clinician education as part of their multicomponent intervention. Fourteen (of a possible 93; 15%) behavior change techniques were identified in the clinician education content of the quality improvement trials, suggesting a focus of addressing the behavioral determinants beliefs about consequences, knowledge, skills, and social influences, of diabetes care providers' behavior. We believe that the Behavior Change Technique Taxonomy version 1 and Theoretical Domains Framework offer a novel lens to analyze the CPD content of existing evidence and inform the design and evaluation of future CPD interventions.

Design principles for professionalizing primary school teachers on promoting reading motivation

Studies show that teachers' continuing professional development (CPD) is essential for educational quality, and moreover, when it comes to reading, key for students' success in education and participation in our 21st-century society. Most of the research investigating professional development programs on improving teachers' self-efficacy for teaching reading and in particular on fostering students' reading motivation, however, fails to include clear and detailed descriptions of the design principles underlying the programs. Therefore, the present study provides a comprehensive description and operationalization of the design principles of a CPD program for primary school teachers focusing on promoting students' reading motivation combining Desimone's (2009) framework for effective professional development with Self-Determination Theory (Ryan & Deci, 2000). Consequently, the CPD program's core features as distinguished by Desimone (i.e., content focus, coherence, active learning, collective participation and duration) and the need for autonomy, competence and relatedness as put central in SDT are analytically described and elaborated on. In view of reporting on the implementation check of the CPD, we further provide insight into whether these operationalized design principles were also perceived as such by the teachers participating in a first iteration of the CPD intervention.

Effectiveness of the spaced education pedagogy for clinicians' continuing professional development: a systematic review.

Ensuring clinical practice reflects current evidence is challenging given the rapid proliferation of new knowledge. Changing entrenched clinical behaviours and facilitating the adoption of best practice evidence requires a range of strategies, including affordable, scalable and effective continuing professional development (CPD). Yet, identifying the CPD delivery method most likely to effectively change and improve patient outcomes is difficult given the variability in the evidence for different learning approaches. Although there is moderate level evidence for outreach education, audit and feedback, and face-to-face or online learning, little is known about the capacity of spaced education to change ineffective clinical practice(s). Spaced education harnesses the power of spacing, repetition and testing learning content to increase topic-specific knowledge. Although spaced education is widely used in undergraduate and postgraduate medical programmes, its effectiveness as a CPD delivery method that improves patient outcomes is less certain. To determine the effectiveness of the spaced education CPD programmes to change targeted clinical knowledge and practice(s) to improve patient outcomes. A systematic review, appraising the spaced education CPD evidence generated from searching six specialist medical and psychosocial databases. Studies published in English peer-reviewed journals from 1 January, 2000 to 31 August, 2018 were eligible for inclusion. A modified Kirkpatrick four levels of evaluation framework assisted with appraising the effect of spaced education CPD interventions on clinicians and patients. Of the 2396 studies identified, 17 met the inclusion criteria, involving 2701 practising clinicians from multiple disciplines and specialties. Five randomised controlled trials generated level II evidence, with the remaining 12 studies generating lower levels of evidence. The majority of studies (n=14) involved the delivery of online spaced education. All studies were evaluated using the modified Kirkpatrick four levels of evaluation framework with: 10 studies demonstrating significant increases in participants' knowledge; seven studies reporting significant changes in clinician behaviour; four studies showing significant increases in clinician confidence; and three studies identifying significant and sustained increases in participants' clinical skills. Only two studies reported positive improvements in patient outcomes. Spaced education via an online platform offers a scalable CPD format that can increase clinical knowledge and change practice. However, further adequately powered randomised controlled trials are required to confirm that spaced education CPD can impact positively on patients' reported outcomes.

Patient Safety and Quality of Care are Everybody's Business: Evaluating the Impact of a Continuing Professional Development Program beyond Satisfaction.

This article was migrated. The article was marked as recommended. Background Research integrating Continuing Professional Development (CPD) with patient safety (PS) and quality improvement (QI) is still in its infancy despite advocacy by leaders in the field. Objectives This theory-driven study explored the feasibility to implement and evaluate the impact of a CPD intervention focused on teaching and practicing PS and QI at the levels of satisfaction, usefulness, knowledge, confidence, intention to change behaviour and reported changes in practice. Methods Three workshops targeting healthcare professionals were delivered live between 2014 and 2016. Data was collected longitudinally through four questionnaires and analyzed with descriptive statistics and triangulation of sources. Thematic analysis of qualitative data was guided by the Theoretical Domains Framework. Results Sixty-seven healthcare professionals participated in the study. Across workshops, satisfaction was high and a significant increase in knowledge and confidence were reported immediately post-intervention. Intention to change behavior was high across workshops. 'Moral norm' and ' beliefs about consequences' were consistently rated as the most influential factors in participants' intention to change behavior while ' social influence' was consistently rated as the least influential. At the workshops, participants anticipated improving communication, increasing their knowledge on PS-QI, applying content learned and building teamwork. Commonly anticipated barriers to implementation included lack of resources, environmental stressors, and the organizational climate/culture. These barriers were confirmed six-month post where participants reported partially implementing 78% (18/23) anticipated goals. Conclusions This study showed the feasibility to develop and implement an effective CPD intervention supporting healthcare professionals' knowledge, confidence, and reported change in teaching and practicing PS-QI.

Effects of physical education continuing professional development on teachers’ physical literacy and self-efficacy and students’ learning outcomes

The purpose of this project is to use a randomized controlled trial to examine the effects of continuing professional development (CPD) on teachers’ physical literacy and self-efficacy and students' physical literacy and physical activity (PA) participation. Seventy teacher participants will be randomly allocated into the CPD intervention and control groups. A total of 50-h of CPD will be completed by teachers during an eight-month period. Teachers will complete physical literacy and self-efficacy questionnaires prior to, and at the end of the CPD program. Following the completion of the PD program, teachers will deliver an 8-month curriculum intervention in schools focused on developing student physical literacy. Prior to, and following the curriculum intervention students will complete questionnaires on their physical literacy, motivation and enjoyment of PA and measured daily PA levels. Teachers’ physical literacy and self-efficacy can be regarded as major determinants of effective physical education teaching, which in turn, influences students’ physical literacy and PA participation.

Designing capacity building of educators in open educational resources integration leads to transformational change

While there is growing recognition and acceptance of Open Educational Resources (OER) and Open Educational Practices (OEP) in teaching and learning, designing for their integration remains very challenging for educators. Adopting OER and OEP in their profession requires significant changes in practitioners’ pedagogical thinking and practices, which can be supported by the design of appropriate professional development experiences. At the Open University of Sri Lanka (OUSL), significant successes in addressing this challenge have been achieved with the design and development of a rigorous approach and a robust model to support adoption of OER and OEP by practitioners using Scenario-based Learning (SBL). In this paper we explore the efficacy of a scenario-based approach to learning in the integration of OER by practitioners, reflecting on how the SBL approach was tested and refined over three continuing professional development (CPD) interventions conducted at the OUSL over 2013–2017. Based on the findings, we discuss what design strategies of the three CPD interventions best supported the adoption of OER and OEP by practitioners, how effective have been the theoretical constructs and the strategies adopted in the design of CPD interventions, the impacts of the CPD interventions and implications of these experiences for the future.

CPD and KT: Models Used and Opportunities for Synergy.

The two fields of continuing professional development (CPD) and knowledge translation (KT) within the health care sector, and their related research have developed as somewhat parallel paths with limited points of overlap or intersection. This is slowly beginning to change. The purpose of this paper is to describe and compare the dominant conceptual models informing each field with the view of increasing understanding and appreciation of the two fields, how they are similar and where they differ, and the current and potential points of intersection. The models include the "knowledge-to-action" (KTA) cycle informing KT, models informing CPD curriculum design and individual self-directed learning, and the Kirkpatrick model for evaluating educational outcomes. When compared through the perspectives of conceptual designs, processes, and outcomes, the models overlap. We also identify shared gaps in both fields (eg, the need to explore the influence of the context in which CPD and KT interventions take place) and suggest opportunities for synergies and for moving forward.

Education and Assessment of Pharmacists on the Use of the Drug Burden Index in Older Adults Using a Continuing Professional Development Education Method

Objective. To educate pharmacists and assess their knowledge of and ability to calculate the Drug Burden Index (DBI) using a continuing professional development (CPD) intervention. Methods. The intervention included designing education surrounding the DBI and its application in practice and assessing knowledge in the form of a CPD education article with four multiple-choice questions (MCQs). Deidentified demographic data on participants were collected. Results. Multiple-choice questions were completed by 2522 pharmacist participants: 97.9% of participants successfully completed the CPD assessment (answered three or four MCQs correctly), and 76.5% of participants achieved a perfect score (answered four MCQs correctly). The question that required calculation of the DBI for a fictional patient was answered correctly least often (81.8%). Conclusion. Pharmacist participants had good knowledge of using DBI in practice; difficulty was observed in calculating the DBI for a hypothetical patient. This CPD intervention provided a practical medium for educating and assessing pharmacists' knowledge of the DBI.