Preceptor Evaluations Research Articles

Development of a collaborative psychiatric advanced pharmacy practice experience

Introduction: Faculty are required to balance multiple duties, including, but not limited to teaching, precepting, and scholarly activity. Likewise, pharmacy managers with faculty appointments are challenged with balancing teaching and administrative duties. With increasing numbers of pharmacy students enrolled in colleges of pharmacy and the recent economic decline, there are concerns that increased responsibilities of faculty and preceptors may lead to sub-optimal advanced pharmacy practice experiences (APPEs). Objectives: Describe a collaborate approach in teaching a psychiatric APPE allowing for additional time for other responsibilities while providing a favorable environment for students. Methods: Pharmacy faculty and managers developed a collaborative psychiatric APPE. The pharmacists divided lectures and case discussions. Students were given many responsibilities (e.g., counseling, intervention documentation). Preceptor evaluations were documented and evaluated. Results: Sixty-four students were precepted over two years. Student evaluations of APPE preceptors averaged 3.74/4.00. Additionally, faculty and pharmacy managers accomplished other activities. Conclusion: It appears this collaborative team taught approach succeeded in providing an auspicious environment for students while allowing preceptors to be productive.

A Comprehensive Approach to Preceptor Development

To assess the impact of a comprehensive preceptor development program. A comprehensive preceptor development program was designed that included live and recorded online programming, a preceptor manual, a preceptor newsletter, live events (local and regional), and one-on-one practice site visits. Over 5,000 evaluations (1,900 pre-implementation and 3,160 post-implementation) of preceptor performance were completed by students. Students rated preceptors higher in items related to providing helpful midpoint and final evaluations after program implementation. Over 1,000 Web-based preceptor development activities were completed by preceptors from 2007 to 2011. Preceptors felt activities enhanced their current knowledge, skills, attitudes, and values, and more than 90% felt the core development activities would improve their current practice. A comprehensive approach to preceptor development that offered a variety of development and training opportunities received positive evaluations from preceptors and resulted in improved student evaluations of preceptors. A comprehensive development program should be made available to preceptors to foster their continuing professional development.

Student performance on and attitudes toward peer assessments on Advanced Pharmacy Practice Experience assignments

PurposeTo compare student (peer) assessment and preceptor evaluation of required rotation assignments and assess students' attitudes toward this process. MethodsFourth-year students completing their Ambulatory Care rotation at Dreyer Medical Clinic from March 2009 to February 2010 were required to complete peer assessments on the rough drafts of three required assignments. Students were trained on proper use of peer assessment forms plus formative feedback questions. After receiving feedback, students had the opportunity to revise final submissions. Preceptors evaluated final versions using the same grading criteria. At the end of the rotation, students provided feedback on the process. A historical control was later created consisting of preceptors' evaluations of students completing their Ambulatory Care rotation at the same clinic the previous year who were not required to complete peer assessments. ResultsTwenty-two students completed the peer assessments. Mean total scores for peer assessments were similar to preceptor evaluations for case presentation (13.2 vs 12.6, P = .209), journal club (12.6 vs 13.1, P = .457), and drug information paper (12.7 vs 12.6, P = .882). Final feedback indicated that students liked receiving their peers' feedback and felt that assessing peers' assignments helped them improve their own assignments. Mean total scores for preceptor evaluations were similar to the historical control for case presentation (12.6 vs 12.8, P = .529), journal club (13.1 vs 13.3, P = .605), and drug information paper (12.6 vs 13.5, P = .121). ConclusionAlthough the opportunity for peer evaluation did not translate into a difference in final grades, it is still recommended based on student feedback and reduced faculty workload.

Preceptored Students in Rural Settings Want Feedback

The purpose of this article is to report the perceptions of students regarding evaluation when placed in a rural setting for their final preceptorship course. The research question was: “What are students’ perceptions of preceptor evaluation?” Thus, a grounded theory method was employed. Twenty-three nursing students placed in rural settings for their precepted practicum experiences were interviewed regarding the evaluation process during their preceptorship experience. The core variable was that feedback is critical for students, much more so than the formal evaluation; and, that students viewed informal evaluation as meaningful and formal evaluation as superficial. The implications are that strategies are required to make students truly authentic partners in evaluation and to make evaluation a useful process in their eyes. The role of the faculty in this process must be reviewed. Feedback is key for students, thus preceptors need educational development in this area.

Association of Third-Year Medical Students' First Clerkship With Overall Clerkship Performance and Examination Scores

Anecdotal experience has suggested that third-year medical students whose first clerkship is internal medicine may have superior performance throughout the academic year. To determine whether the order of clerkships by specialty is associated with student performance. Clerkship performance records of medical students at all 4 campuses of the University of Illinois College of Medicine who completed their third-year core clerkships from July 2000 through June 2008 (N = 2236) were reviewed. Analysis of covariance was used to test for between-group differences (by first clerkship) in mean National Board of Medical Examiners subject examination scores (range, 0-100), preceptor ratings of clerkship clinical performances (range, 12-30), total overall clerkship grades (range, 12-30), and United States Medical Licensing Examination (USMLE) Step 2 scores, adjusted for sex, campus, and USMLE Step 1 score. First clerkship specialty was significantly associated with mean subject examination scores (family medicine, 71.96 [95% confidence interval {CI}, 70.90-72.98], internal medicine, 73.86 [95% CI, 73.33-74.39], obstetrics/gynecology, 72.36 [95% CI, 71.64-73.04], pediatrics, 73.11 [95% CI, 72.38-73.84], psychiatry, 72.17 [95% CI, 71.52-72.81], surgery, 72.37 [95% CI, 71.73-73.02]; P < .001) and overall clerkship grades (family medicine, 24.20 [95% CI, 23.90-24.90], internal medicine, 25.33 [95% CI, 25.07-25.60], obstetrics/gynecology, 24.68 [95% CI, 24.32-25.05], pediatrics, 24.92 [95% CI, 24.59-25.27], psychiatry, 24.61 [95% CI, 24.33-25.01], surgery 24.97 [95% CI, 24.64-25.30]; P = .01). There was no significant association with preceptor ratings or USMLE Step 2 scores. Pairwise comparisons for mean total overall clerkship grades showed a significant difference for students taking internal medicine first compared with obstetrics/gynecology (mean difference, 0.65; 95% CI, 0.18-1.12), psychiatry (mean difference, 0.66; 95% CI, 0.20-1.12), and family medicine (mean difference, 0.93; 95% CI, 0.37-1.50). Among students at 4 campuses of a US medical school, clerkship order was significantly associated with performance on clerkship subject examinations and overall grades but not with clerkship clinical performance or USMLE Step 2 scores.

University of Alabama School of Medicine

University of Alabama School of Medicine

Clinical evaluation tools for dual track adult and gerontology nurse practitioner students

To describe the development of a dual track offering for the Adult Health and Gerontology Nurse Practitioner (AGNP) Programs at the University of Pennsylvania School of Nursing and share clinical evaluation tools used with nurse practitioner students in this curriculum. Selected research and clinical articles. A variety of evaluation approaches are utilized in the AGNP student clinical performance evaluation. These incorporate the extended clinical practicum sequence for the dual track curriculum in addition to each individual program's objectives. Formative and summative evaluations include reflective logs, clinical documentation of patient encounters, preceptor evaluation, and faculty site visits. Self-evaluative skills of the student and quality faculty feedback are two additional integral components that facilitate learning outcomes in the cognitive, psychomotor, and affective learning domains. The summary of evaluation tools presented here is an example of how the AGNP Programs at the University of Pennsylvania School of Nursing effectively measure student progress in a curriculum model for dual track enrollment.

Objective Structured Clinical Examination and Advanced Practice Nursing Students

Educators, challenged to measure clinical competency impartially in Advanced Health Assessment courses, have used Objective Structured Clinical Examination (OSCE) and standardized patients (SP). Faculty-trained laypersons act as patients (SP) for students in a clinical laboratory setting using a standardized, scripted scenario. Students typically are evaluated during the examination and receive instant feedback from the SP. There has been little evidence that supports this as the best way to measure student's clinical competency. This study's purpose was to compare outcomes of graduate nursing students completing a traditional methods Health Assessment course to those students using OSCE and SPs. Problem-based Learning Theory guided the quasi-experimental study that included 37 students divided into research and control groups. There were statistically significant differences between the two groups for the final practical examination grades, clinical preceptor evaluations, satisfaction scores, and self-evaluations of skills at the course's end. Research group's course scores were higher than the control group. There was no difference between group's self-evaluation for their current assessment skills. This intervention had a positive impact on students' outcomes. Educators should incorporate SPs and OSCE to improve clinical competency scores, course satisfaction, and preceptor evaluations. Future studies should include a longitudinal design and qualitative student feedback.

Attributes of effective community preceptors for pre-clerkship medical students

Background: Attributes of effective precepting of pre-clerkship medical students in community settings are not adequately described. As part of preceptor needs assessment, we conducted site visits over three consecutive years. We also measured the preceptorships’ educational quality, using students’ post-rotation data, to identify priority areas for faculty development, corroborate site visit findings, and assess functions of the site visit.Methods: Three university faculty directly observed teaching encounters in 83 community preceptors’ offices during a 12–18-week second year (pre-clerkship) medical student rotation. Data were collected on practice demographics, teaching content, and educational quality, using multiple measures. Narrative responses to interview questions were coded for prevalent themes. Student post-rotation assessments were obtained by anonymous online evaluations.Results: Good precepting attributes and suggestions for improvement were identified from both narrative analysis and student post-rotation evaluations in these key areas: independence in patient assessment, time spent teaching, giving feedback and orientation to the preceptor's practice. Student evaluations of preceptors’ effectiveness significantly improved from year 1 to 2 and persisted into year 3. Appropriate faculty development strategies were derived from the combined results.Conclusions: The site visit by university faculty allows real-time observation and may itself be an effective intervention for improving teaching and learning.

Evaluating the impact of a pre-rotation workshop on student preparation for clinical advanced pharmacy practice experiences.

This pilot study was designed to evaluate the impact of a pre-rotation workshop (PRW) on pharmacy students' clinical skills and preparation for clinical Advanced Pharmacy Practice Experiences (APPE) involving direct patient care. Randomized controlled trial of an educational intervention with Institutional Review Board approval. PRW activities designed to simulate rotation activities around five competencies, patient charts, medication histories, SOAP notes, patient presentations, and professionalism. Endpoints were evaluated using clinical rotation preceptors' evaluation of performance and students' performance on objective structured clinical exams (OSCE). Eight fourth-year students and eight GPA matched controls (20% of the total class) were selected to voluntarily participate. The PRW demonstrated a positive impact on students' clinical skills and preparation for rotations by improving OSCE performance. However, no significant differences were found between groups when comparing preceptor evaluations of skills on rotations. These results are limited by the small sample size, potential OSCE "test-wiseness" effects, lack of OSCE evaluator blinding to study groups, potential case specificity effects due to the limited number of cases used on the OSCE and possible lack of sensitivity of the rotation evaluation tool to capture true differences among the experimental and control group participants. The PRW was successful at advancing students' clinical skills and preparation for rotations and may be considered as a tool to help bridge didactic to clinical experiences in the Pharm.D. curriculum.

Clinical Preceptor Evaluation of Veterinary Students in a Distributed Model of Clinical Education

The study reported here investigates the reliability and validity of a standardized evaluation form used to assess students' knowledge, clinical skills, interpersonal skills, and professionalism during fourth-year clinical rotations in a distributed model of veterinary education. A form designed to assess veterinary knowledge (5 items), clinical skills (7 items), interpersonal skills (3 items), and professionalism (6 items) was used by clinical preceptors to evaluate student performance across different rotations. For the period January--May 2007, 218 evaluations were completed for 81 students; each student was assessed in at least two rotations. Mean scores across the 21 items ranged from 3.42 (SD = 0.61) to 3.87 (SD = 0.37). Construct validity was assessed using exploratory factor analysis. The 21 items loaded on three underlying factors, professionalism, knowledge and clinical skills, accounted for 70.35% of the variance. Internal consistency (Cronbach's alpha) of each subscale was high, ranging from 0.88 for clinical skills to 0.94 for professionalism and 0.96 for the entire tool. Correlations between subscales were significant (p < 0.01), ranging from r = 0.62 to r = 0.76. Preliminary analysis suggests that the evaluation tool has good internal reliability. Construct validity analysis suggests that certain items relating to interpersonal skills and clinical skills were assessing either knowledge or professionalism. Clinical preceptors could differentiate between different skill levels for knowledge and clinical skills. Challenges associated with the assessment of professionalism are discussed.

Developing a Practical Evaluation Tool for Preceptor Use

After years of dissatisfaction with existing instruments, a tool for preceptors to evaluate an undergraduate student's clinical performance was developed, with preceptors' input in its construction. A 2-year pilot evaluation revealed notable problems including excessively high preceptor ratings and significant disparities between faculty and preceptor ratings. Further revisions were made, reducing indicators to those which the preceptors can actually evaluate on an everyday basis and developing a rubric. Additional recommendations to bolster the quality of ratings are improving orientation and guidance of preceptors and modifying procedures for giving feedback.

An Association Between Paying Physician-teachers for their Teaching Efforts and an Improved Educational Experience for Learners

Medical schools often rely on faculty volunteerism to address clinical teaching needs for students. Increasing time pressures on physicians has made it difficult to secure commitments for clinical instruction. In the 2005-2006 academic year, the Johns Hopkins University School of Medicine (JHUSOM) launched the Colleges Program, recruiting 24 salary-supported physician-faculty to serve as advisors to students as well as teachers of the second year course, 'clinical skills'. We hypothesized that compensating physician educators would have a measurable positive impact on the students' experiences in this course. Students' assessments of paid colleges faculty (CF) preceptors from the 2005-2006 year were compared to those of volunteer preceptors from the two prior years (2003-2005 academic years) along six different teaching parameters linked to the course's objectives. Multivariable regression analysis was used to identify the factors independently associated with higher preceptor scores. Fifty-eight preceptors taught clinical skills over the 3-year study period. The overall response rate for preceptor evaluations by medical learners was 77% (277/359). CF, more likely than volunteer preceptors to have a full-time academic appointment (100 vs 63%, p < .01), have an additional advanced degree (48 vs 15%, p < .01) and prior faculty development training (52 vs 17%, p < .01). Scores for all six evaluation domains were higher for CF compared to those from the two previous years combined (all p < .001). In the fully adjusted regression model, only CF status was independently associated with high preceptor evaluation scores (Odds Ratio 4.3, 95% CI 1.01-18.20). Salary support for teaching efforts in the time-intensive CS course coupled with the prestige of being appointed to the CF was associated with higher student evaluations.

Faculty Q&amp;A

&lt;table align="center" border="1" cellpadding="8" cellspacing="0" width="550"&gt; &lt;tbody&gt;&lt;tr&gt; &lt;td&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;Send in your questions for Faculty Q&amp;amp;A!&lt;/h3&gt; &lt;p&gt;We want your questions not only about issues related to how to teach and instructional strategies, but also about other issues faced by faculty (both new and established). Here’s an example: &lt;/p&gt; &lt;p&gt;“I was shocked to have a student accuse me of sexism. The student is male and says there is an inherent bias against male students, as I have answered questions posed to me by female students when we were in the change room preparing for a clinical shift. How can I guard against this type of unintentional problem in the future?” &lt;/p&gt; &lt;p&gt;&lt;b&gt;Here’s what you need to know: &lt;/b&gt;&lt;/p&gt; &lt;ol&gt; &lt;li&gt; Questions need to be short and, preferably, include a specific scenario or examples.&lt;/li&gt; &lt;li&gt;Submit questions that address practical issues faced by faculty and can be answered in a few paragraphs (see example above).&lt;/li&gt;&lt;li&gt;E-mail your questions, along with your full name and credentials, to Karen G. Stanwood, ELS, Executive Editor, at &lt;a href="mailto:kstanwood@slackinc.com"&gt;kstanwood@slackinc.com&lt;/a&gt;.&lt;/li&gt;&lt;/ol&gt;&lt;/br&gt;&lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt;&lt;/table&gt; &lt;h4&gt;EXCERPT&lt;/h4&gt; &lt;P&gt;With the increase in numbers of online and distance-learning master’s and doctoral programs for nurses, the faculty in our moderately small baccalaureate and RN-to-BSN completion program are being swamped with requests to be preceptors for local nurses needing close-to-home teaching and related field experiences as a component of their educational programs. Although we want to support the continuing education of these nurse colleagues, we do not have the resources to precept them in addition to our own teaching loads and other college obligations. How do you suggest we handle such requests without alienating future nurse educators, thereby possibly “cutting off our own hand” as the need for qualified nursing faculty continues to grow? We want to support their education but not at the risk of failing to meet the needs of our own student population.&lt;/P&gt; &lt;P&gt;I teach in a graduate program that prepares students to be nurse practitioners and clinical nurse specialists. Faculty members visit the students in their clinical sites twice during the semester. Students receive a grade for their clinical practice on the basis of preceptor evaluation, faculty evaluation, and students’ self-evaluation. Although we have spent a great deal of time as a faculty on developing interrater reliability using the clinical evaluation tool, I’ve noticed that almost all of the students receive a grade of “A” in their clinical. Faculty members are unwilling to move to a pass/fail clinical evaluation, and it seems clear that there is significant grade inflation. How can we better differentiate between the A, B, and C student and address the grade inflation issue? &lt;/P&gt; &lt;P&gt;My faculty is in the process of a major revision of the BSN curriculum. In reviewing the program objectives, several faculty members commented that the objectives should be rewritten to reflect outcomes not objectives. This has led to some debate with no resolution. I have always used the term objectives prefaced by the statement “by the end of the course (or program) the student will be able to _____.” Then in my course, I have used smaller, sub-objectives to detail what the student will be able to do at the end of a unit of study during the semester. Should we be using the term outcomes? Is there really a difference between outcomes and objectives?&lt;/P&gt;

Medical student evaluation of teaching quality between obstetrics and gynecology residents and faculty as clinical preceptors in ambulatory gynecology

This study was undertaken to compare teaching quality between obstetrics/gynecology resident and faculty preceptors in ambulatory gynecology as determined by medical student evaluation. A prospective assessment of medical student evaluations of resident and faculty preceptors in ambulatory gynecology was conducted at Brigham and Women's Hospital. Forty-eight third-year medical students evaluated preceptors on 15 teaching quality items and recorded their clinical experience during their obstetrics/gynecology clerkship from July 2004 to June 2005 for a total of 138 evaluations. Faculty scores were statistically higher than resident scores on 4 of the 15 teaching quality items; acting as an appropriate clinical role model, emphasizing evidence-based learning, being enthusiastic about teaching and patient care (P < .05). Students performed significantly more Papanicolaou tests and pelvic and breast examinations during ambulatory sessions with residents than with faculty (P < .001). Residents and faculty contribute important and different aspects of teaching experiences for medical students in ambulatory gynecology.

Should faculty make clinical site visits for students or rely on clinical preceptor evaluation?

Should faculty make clinical site visits for students or rely on clinical preceptor evaluation?

APPLIED RESEARCH: Standardized Versus Real Hospitalized Patients to Teach History-Taking and Physical Examination Skills

Background: Despite the nearly universal practice of using standardized patients in introduction to clinical medicine (ICM) courses, no studies have compared the performance of students trained with standardized patients to that of those trained with hospitalized patients with regard to short- and long-term educational outcomes.Purpose: To examine the differential effect of the use of standardized patients in a simulation center versus the use of hospitalized patients in affiliated teaching hospitals for teaching history-taking and physical examination skills.Methods: This was a nonrandomized cohort study based on self-selection involving students from 2 academic years enrolled in an ICM course who received the final block of their ICM instruction with either standardized patients in the simulation center or hospitalized patients in affiliated hospitals. The primary, end-of-ICM (preclerkship)-outcome variables (k = 10) were data from a final observed history and physical examination, an observed structured clinical examination, a National Board of Medical Examiners subject examination for clinical medicine, and ICM preceptor evaluations. Secondary, postclerkship outcome variables (k = 5) included internal medicine clerkship scores, teacher ratings from the clerkship, and examination scores from end-of-clerkship tests. The statistical significance was set at p <. 05.Results: No statistical differences were found between the means of the 2 groups in any of the primary or secondary outcome variables; there was significant power.Conclusions: The use of standardized patients in a simulated setting compared to the use of hospitalized patients in affiliated teaching hospitals is not a disadvantage in the education of students in ICM courses.

An Evaluation of Preceptors’ Perceptions of Educational Preparation and Organizational Support for Their Role

This longitudinal, descriptive study evaluated nurse preceptors' perceptions of a 2-day educational workshop and subsequent organizational support offered to prepare them for their roles as preceptors. Thirty-six registered nurses who attended preceptor workshops and then acted as preceptors participated in focus groups or interviews 2 to 3 months and 6 to 9 months after their educational preparation to identify the usefulness of the preparation, the subsequent support by management, and the potential changes in their perceptions. Respondents identified intrinsic rewards and opportunities for growth for nurses who take on the preceptor role. However, adequate preparation and organizational support are essential to effectively perform their duties as clinicians and preceptors. Although a 2-day educational preparation program was perceived as adequate, the practice setting must be reorganized to provide greater support for preceptors. Preceptors should be supported by providing continuing education, effective scheduling, and adequate time for learning and feedback in the clinical environment.

Faculty Clinical Site Visits Versus Preceptor Evaluation of NP Students

In 2002, the National Task Force on Quality Nurse Practitioner Education, in the Criteria for Evaluation of Nurse Practitioner Programs, published the mandate to “evaluate students cumulatively based on clinical observation of student performance by NP faculty and the clinical preceptor's assessment” and that “Direct clinical observation of student performance is essential.” The National Organization of Nurse Practitioner Faculties (NONPF) supports this mandate, including for distance education students.

Relationship among scores of medical student assessments in a new integrated curriculum

Purpose. The medical curriculum at the China Medical University, Taiwan was reformed in 2001. A new integrated curriculum (New-IC) that includes organ-based block, problem-based learning (PBL) and clinical skills and communication (CSC) courses has replaced the traditional discipline-based, lecture-intensive coursework. Multiple assessment instruments including multiple-choice questions, direct observation and preceptor ratings were used accordingly. This study describes our new curriculum, examines the results of student assessments and tries to explore the relationship among these assessments. Methods. The scores of two student cohorts obtained from written exams, PBL and CSC including history taking, physical examination and clinical procedure were calculated during one academic year. The relationship among these scores within each organ-based block was analyzed by Pearson correlation. Results. A total of 243 students who completed 12 blocks were assessed during the 2003 to 2004 academic year. Although there were some statistically significant correlations between the scores of different assessments, in general, the correlation coefficients in most cases were quite low, ranging from -0.233 to 0.377. Conclusions. Assessments used to evaluate knowledge, skill, self-directed learning and attitude in our New Integrated Curriculum were not correlated with each other. All instruments have their own strengths and weaknesses; therefore a good assessment program requires a variety of assessment instruments, with each one designed to discover something unique.