Becoming a better preceptor: The evaluation process

Abstract

Figure. ©: iStockphoto.com/WarchiEvaluation is not just a static event done at the end of clinical rotation. It is a dynamic process that begins with providing well-written learning objectives at the start of the rotation, and concludes with sharing observations of the progress that has been made. Between these endpoints, the preceptor provides ongoing feedback, preferably daily, to facilitate modification and improvement of the student's clinical performance. When the evaluation process is performed appropriately, the quality of the learning experience is enhanced for the student and the preceptor, and the stage is set for future professional growth. Evaluation is not just a grade assignment. It is a process for guiding and contributing to the clinical and professional growth of our future colleagues. MEASUREMENT VS. ASSESSMENT While we often use measurement, assessment, and evaluation interchangeably, each term has a different meaning and function. Measurement is the act of gathering information, while assessment determines whether the student has acquired a specific set of skills or knowledge. The preceptor is continuously assessing the student's current knowledge by observing him and providing ongoing feedback. Evaluation, on the other hand, is collecting information and placing a value on it. Assessments serve as the basis for evaluation. Assessing and evaluating students' skills must be based on predetermined learning objectives—specific, measurable, and clearly written statements of the expected outcome for a given clinical experience. Learning objectives provide the foundation of the evaluation process and facilitate discussions of the student's past experience, strengths, and areas for further development. For the student, these objectives provide expectations for what should be achieved by the end of the rotation. For the preceptor, they serve as the basis for what will be assessed and evaluated during and at the conclusion of the rotation. They also guide the preceptor in making his teaching more focused and organized. When writing learning objectives, each objective must address four components: Who is the audience? What is the activity? When will the activity be completed? How well will the activity be performed? Failure to include any of these components results in a unmeasurable objective which, consequently, is of no value to the student or preceptor. Every rotation and every preceptor should have a set of effective learning objectives that are discussed at the beginning of the rotation, reviewed throughout it, and serve as the benchmark for the evaluation. The objectives need to be: Consistent with the goals of the academic institution. Clearly stated, consisting of the four critical elements. Measurable. Appropriate for the skill and knowledge level of the student. Students who receive ongoing feedback specific to and focused on behavior and performance have better outcomes than students who receive little or no feedback. FEEDBACK Preceptors should provide informed, non-evaluative, objective appraisals of performance based on ongoing observations of the student. Feedback is an essential and critical component in the evaluation process but often overlooked by preceptors. When given daily, it provides support and guidance for the student. When little or no feedback is provided, mistakes go uncorrected and students can only gauge their performance by trial and error.1 Without appropriate feedback, good performance is not get reinforced, and clinical competency may be achieved empirically rather than systematically. Specific criteria provide effective feedback. (Table 1.) Feedback should be delivered nonthreateningly, directed toward specific behavior, and provided in a timely fashion. Feedback can be positive and reinforcing, or it can be corrective, with the goal of changing the student's behavior, skills, knowledge, and performance. An interesting model for providing feedback is the Ask-Tell-Ask model.2 This method can be employed for immediate, daily feedback or as part of the review process, and takes less than five minutes to complete. (Table 2.)Table 1: Effective feedback should.Table 2: Example of the Ask-Tell-Ask feedback model.Research shows that students who receive regular feedback significantly improve their performance, develop better judgment, and learn faster than those who do not receive adequate feedback.3-5 Perhaps surprisingly, providing feedback is rated among the most important qualities that make a good preceptor.6,7 Ongoing formative assessments provide a guide to the student regarding his progress, accomplishments, and any areas that require improvement. Students should be encouraged to conduct self-reflection of strengths and weaknesses during the formative assessment. Because these assessments are conducted during rotation, students should have sufficient time to modify their behavior to improve their skills. The assessments should also encourage the preceptor to evaluate his own teaching methods, altering them as needed. Remember that the intent of formative assessment is to provide suggestions and methods to improve clinical performance and knowledge, not to provide a grade. Summative evaluation, however, is a formal method used to summarize the student's performance and provide a grade. These evaluations should also continue regularly to indicate clearly the student's strengths, weaknesses, and opportunities for improvement. ASKING QUESTIONS One of the most effective teaching methods in the clinical setting is asking appropriate questions at opportune times. Questioning can determine students' needs, promote curiosity, stimulate the ability to think critically and solve problems, and model essential professional behavior. Yet not all questions are created equal or serve the same purpose. A number of factors should be considered when formulating questions. Ask different types of questions. Closed questions are useful in recalling facts, prioritizing processes, and challenging the student's ideas, thoughts, or actions with the expectation of only one correct answer. An example of a closed question is, “What is the generator site for wave I of the ABR?” Open or divergent questions foster reflection or speculation with the possibility of more than one correct answer. Open questions require higher level cognitive performance, which may facilitate problem-solving, and are most effective in encouraging discussion or more active learning. Open questions can clarify answers; explore attitudes, values, or feelings; reveal alternative options or perspectives; investigate thought processes; or request justification or support of answers or interpretations. An example of an open question is, “What sound therapy options might be helpful for this patient suffering from tinnitus?” Ask questions that vary in function. Assess knowledge by using what, when, and where types of questions. Asking questions about how something is done will allow preceptors to assess the student's ability to apply theoretical knowledge in the clinical setting. If the goal is to determine whether the student understands certain concepts, preceptors should ask why actions were taken. Ask questions that are appropriately complex. Questions should vary in their level of complexity and parallel Bloom's taxonomy.8 Asking students to recall facts such as listing the symptoms consistent with Ménière's disease, is appropriate for beginners. In contrast, asking a student to justify his recommendations for a given patient involves a much higher taxonomy level. Ask only one question at a time. Don't confuse the student by asking a series of questions all in one breath. Allow time for the student to respond to the first question. Try rephrasing the question if you fail to get a response. Allow opportunities when the student fails to answer. Failing to answer a question can be an excellent active learning experience because it places the responsibility back on the student to research the answer. Ask direct, clear, and specific questions. Use a sequence of questions to build depth and complexity rather than use a single question that is multilayered and complex. Follow poor or incomplete answers with another question. Again, build on the answer with questions that may be different in complexity, type, or intent. Use lower-order thinking and closed questions to assess knowledge and comprehension. Use higher-order thinking and open questions to assess the student's ability to apply, analyze, synthesize, and evaluate. Wait at least three seconds after asking a question before verbalizing again.9 In teaching situations, silence tends to make us uncomfortable. Yet silence allows the student to formulate an answer that tends to be longer in length, more elaborate in content, and better supported by literature. When asking a question, maintain a noncommittal stance. Be aware of what your body language is saying to the student. Ask questions with your arms uncrossed, and lean forward when seated to show you're engaged. Make sure your facial expressions also show you are listening. Model questions. When asking yourself questions as you care for your patients, ask those questions out loud. The student benefits from role modeling as you proceed through your decision-making process. BIAS ERRORS A variety of errors may inadvertently occur in your assessment of the student's performance. Be aware of potential biases or sources of error that exist when evaluating students, and regularly question yourself to verify that you have not allowed any of these to occur. The Halo Effect occurs when one positive (or negative) characteristic of the student influences or supersedes another. For example, Joe is given a higher grade despite his mediocre clinical skills because he is such a team player and has such an outstanding personality. Memory Lapse happens when preceptors cannot remember details of an event that happened earlier in the rotation. Carrying a notebook to jot down examples or comments is a fairly simple method to ensure feedback contains specific examples that can be recalled. The Leniency Error results when everyone gets higher grades independent of clinical performance because the preceptor wants to promote himself, thinking the students will like him if he gives all As, or to prevent student conflict to avoid having to defend the grade or deal with unhappy students by giving all As. The Recency or Primacy Error occurs when the most recent performance influences the grade rather than taking into account total performance throughout the rotation. The Past Performance Error occurs when assessment is based on performances earlier in the rotation rather than the entire rotation. The Similar to Me and the Different From Me Errors come into play when the preceptor relates to the student or has similar values and evaluates that student higher than appropriate. Likewise, a student may be more harshly assessed if he is perceived to have different values from or a personality clash with the preceptor. Lake Wobegon happens when the preceptor views all students as essentially equal in their clinical competency. Students who are truly outstanding do not get acknowledged or rewarded, and weaker students are not identified and subsequently not provided remediation. OBSERVATION As preceptors, we observe many aspects of our students' behaviors in and out of the clinic. We tend to observe behaviors that are more dynamic, expected, or most valued.10 By defining the behavior that will be the focus of the observation, you can concentrate on that particular event rather than attempting to critique the entire process. (Table 3). After reviewing the day's patient list and noting that five patients will be seen for audiologic evaluations, for example, prepare an observation schedule grading the student's timeliness in greeting the patient, ability to bracket and obtain pure tone air and bone conduction thresholds, and ability to obtain a seal to perform tympanometry. At the end of the day, you will have observed key behaviors and provided systematic feedback to the student. When the rotation ends, you will have assessed all behaviors and competencies dictated by the learning objectives.Table 3: Tips for conducting a quality observation.ORIENTATION Conducting a systematic orientation at the outset of the clinical placement is vital to the success of the experience. Orientation provides an opportunity for the student to become familiar with the physical layout of the office, to learn how things operate, and to understand the philosophy of the practice. Orientation can be conducted on the first day or within several days of the rotation, and should address a number of topics related to their responsibilities, your expectations, and practical office rules. Learning contract During orientation, a learning contract should be presented and discussed. The negotiated agreement between the student and preceptor provides: Expectations for each individual rotation, if applicable, or the entire clinical rotation at your site. Learning objectives. Opportunities for a discussion of strengths and weaknesses. By accepting students into your clinical setting, you create learning opportunities simply by immersing them in the art and science that defines our contemporary professional scope of practice. Your participation as a clinical educator is a collaborative relationship with the student, his university program, and of course, patients seeking hearing health care. For the experienced preceptor, we hope this series of articles (http://bit.ly/PreceptorPart1; http://bit.ly/PreceptorPart2; http://bit.ly/PreceptorPart3); has reinforced and expanded your skills and knowledge about precepting. For those of you getting your feet wet, don't be afraid to take the plunge. With proper motivation and some initial thought and preparation, you should anticipate finding the precepting experience challenging, energizing, rewarding, and hopefully fun.