Tutor Evaluation Research Articles

Community-oriented medical emergency programme: development and evaluation issues

A community-oriented medical emergency programme for multidisciplinary healthcare professionals on the Higher Professional Diploma in International Healthcare Studies, at the United Medical Education College, London is described. The main aim of the course is to introduce students to the clinical skills of dealing with medical emergency problems and situations. Students are also exposed to how medical emergency is practised in the community. The four-week programme consists of 16 sessions of clinical skills teaching where students are attached to various community clinics, 10 sessions of college teaching, consisting of talks, workshops and interactive small-group work and four sessions encouraging students to learn independently in a self-directed learning format. Student and tutor evaluations of the programme demonstrate that its aims are being met. However, further adaptation of the module to increase the range and quantity of clinical cases for student learning is being planned.

A process evaluation of medical ethics education in the first year of a new medical curriculum.

Despite the recent increase in activity in the field of medical ethics education, few evaluative studies have been carried out. Most studies have taken place in North America, in curricula where teaching is discipline-based, and have concentrated on outcome rather than on the curricular processes adopted. To evaluate the process of medical ethics education in the first year of a new learner-centred, problem-based, integrated medical curriculum. A qualitative, multi-method approach was adopted using open questionnaires, focus groups and tutor evaluation rating scales. The study involved all 238 students in the first year of the new medical curriculum, and the 30 clinical tutors who facilitated ethics learning. A stratified sampling technique was used to choose focus group participants. Small group teaching proved highly acceptable to both students and tutors. Tutors' teaching skills were central to its effectiveness. Tutors played an important role in promoting students' appreciation of the relevance of medical ethics to clinical practice, and in establishing a climate where constructive criticism of colleagues' actions is acceptable. Course integration, including the provision for students of clinical experiences on which to reflect, was an important aid to learning. Students and tutors were noted to be driving the ethics curriculum towards having a contextual rather than theoretical base. This evaluation identified those aspects of the medical ethics course which contributed to its effectiveness and those which detracted from it. This information will be used to inform future development.

'Let's You and Me Have a Little Discussion': Computer mediated communication in support of campus-based university courses

Computer mediated communication (CMC) is increasingly used in support of campus-based university courses. This study focuses upon student-student interaction via CMC in the context of an undergraduate course. The tutor provided the initial structure for the interaction but did not contribute to the on-line discussion at all. Data were collected by individual and group interview, and the electronic exchanges were gathered and analysed. Analysis focused upon (i) patterns of interaction in the various groups, (ii) the influence of the tutor's 'virtual absence', and (iii) the students' and tutor's evaluation of the experience. Some students were repeatedly subjected to on-line criticism by peers. However, most felt that both the CMC experience and the requirement to evaluate it had given them a valuable opportunity to reflect on this approach to learning.

How to communicate politely and be a tutor, too: NS-NNS interaction and writing center practice

Conflicts between comprehensibility, politeness, and effective tutorial practice frequently create communicative impasses in interaction between university writing center tutors and their tutees, who are both native (NS) and non-native speakers (NNSs) of English. Conversational excerpts drawn from a study of 34 tutorials demonstrate that such impasses are more frequent in tutorials with NNS tutees. The primary context for these impasses is the offering of tutor evaluations and suggestions, although the interactional features of volubility and interruptions are also implicated in the analysis. I argue that the trade-offs tutors make between comprehensibility, politeness, and effectiveness are essential characteristics of NS-NNS tutorial interaction and that open acknowledgment of the conflicts between communicative aims is essential to principled tutorial practice in this institutional discourse context.

Self and tutor evaluations in problem-based learning tutorials: is there a relationship?

It is now recognized that acquiring specific skills in self-evaluation helps students in a more general appraisal of their overall performance. Self- and peer-evaluation skills are essential prerequisites for the success of every doctor in maintaining professional competence. In the Faculty of Medicine and Health Sciences (FMHS) of the United Arab Emirates, problem-based learning (PBL) is instituted in the first year. Self-evaluation by students and tutor rating of students' performances are an integral part of the PBL tutorials. This has provided the opportunity to conduct a systematic study of the role of self-evaluation by students as distinct from the tutor evaluation of students in the PBL tutorials for five themes. The study sample included all preparatory year (first-year) students who joined the FMHS in 1994 and 1995. A total number of 64 students participated, of whom 17 (26%) were male and 47 (74%) female. Mean self-evaluation scores were high throughout the module and did not follow any trend from theme one to theme five. While self and tutor scores were similar, male student self-evaluation scores were higher than for female students on overall scores. The sharing of assessment reports between students and tutors has been perceived to be a useful tool for the students' development of the skills of analysis, differentiation and critical appraisal.

Students' communication apprehension and its effects on PBL performance

SUMMARY This study examined the relationship between medical students' communication apprehension and their performance in problem-based learning (PBL) sessions as assessed by the tutor and by their knowledge acquisition. At the beginning of the third-year surgery clerkship, the Personal Report of Communication Apprehension (PRCA-24) was administered to 85 medical students. The PRCA-24 scores of students in the PBL curriculum were correlated with two outcome-based measures: evaluations by faculty tutors of the students' performance in the PBL sessions, and a modified essay examination designed to measure students' knowledge derived from PBL sessions. Students with high levels of communication apprehension had significantly greater difficulty participating in PBL sessions, as shown by their tutor evaluations. However, their levels of communication apprehension did not correlate with their performance on the modified essay examination. These findings indicate that students' apprehension about communication in group settings affects PBL tutor evaluations of their performance.

Test-Retest Reliability of a Tutor Evaluation Form Used in a Problem-Based Curriculum

This paper reports an investigation of the test-retest reliability of student evaluations of tutors in a Problem-Based Learning curricula at McMaster University. Thirty first-year Occupational Therapy students in five tutorial groups completed a 19 item tutor evaluation scale. To examine test-retest reliability they completed the scale on two occasions separated by a three day interval. Fourteen items had test-retest generalizability coefficients greater than .70. Student evaluations appear to have promising reliability. Suggestions for the improvement of reliability include familiarization with the form and ensuring that administration occurred separately from formal student evaluations.

An investigation of a tutor evaluation scale for formative purposes in a problem-based learning curriculum.

An investigation of a tutor evaluation scale for formative purposes in a problem-based learning curriculum.

Long-term stability of tutor performance.

The aim of this study was to investigate to what extent ratings of tutor performance remain stable in the long term. At many schools, teaching performance is assessed and these evaluations are consulted as part of the decision-making process for promotion, tenure, and salary. Since this information may have summative value, it is crucial that the reliability of the data be assessed. A previous study had shown that a single evaluation of a tutor is reliable when the responses of six students are used (interrater reliability). The present study focused on the stability of tutor evaluations over repeated occasions of evaluation. A generalizability study was conducted to estimate the number of occasions required to demonstrate stability. The study took place during three academic years (1992-93, 1993-94, and 1994-95) at the problem-based medical school of the University of Limburg (now Maastricht University). A total of 291 ratings were analyzed (97 tutors rated during three sequential tutoring occasions). Two types of scores were used: an aggregate score calculated from ratings of 13 items and an overall judgment. The results indicate that when the scores are used to interpret the precision of individual scores, two evaluation occasions should be available for the overall judgment and four occasions for the aggregate score. If the tutor scores are consulted only to determine whether performances are above or below a cutoff score, a reliable decision can be made after only a single occasion of evaluation. The results demonstrate that data collected over an extended period of time can be reliably used as part of the decision-making process for promotion, salary, and tenure.

Development and validation of an evaluation instrument for medical students in tutorials.

To develop and explore the validity of the Tutotest, the first standardized instrument designed to assess the skills and attitudes of medical students working in tutorials in a problem-based learning curriculum. The Tutotest, consisting of 44 items rated on a four-point Likert-type scale, was developed from well-defined theoretical frameworks after consultation with students and teachers at the Université de Sherbrooke Faculty of Medicine. In 1988-89, 40 tutors were asked to use the Tutotest to evaluate a total of 100 students at the end of each of the four units of the first-year curriculum. Data were analyzed to examine the homogeneity and reliability of the instrument, explore its factorial structure, and document its correlation with students' grades. A total of 28 tutors (70%) conducted Tutotest evaluations; of a possible 400 evaluations, 270 (67.5%) were returned with complete data. Exploratory factor analysis revealed that the Tutotest was structured around four factors that accounted for 82% of the variance: effectiveness in the group (23 items), communication and leadership skills (13 items), scientific curiosity (four items), and respect for colleagues (four items). The Tutotest had a Cronbach's alpha coefficient of .98 and a split-half coefficient of .98. The intraclass correlation coefficient was estimated to be .46 and would reach up to .81 after averaging five Tutotest evaluations. Tutotest scores showed good correlation (r = .64) with the official tutor's global evaluation and a moderate correlation (r = .39) with students' written examination results. The Tutotest was found to be a standardized reliable and valid instrument that can significantly improve the evaluation of students' skills and attitudes during tutorials. Although lengthier than the usual tutor evaluation form, the Tutotest is a good compromise between an unreliable short global scale and a more comprehensive but impractical instrument.

Balancing service and education: improving internal medicine residencies in the managed care era

Internal medicine training programs must adapt to health care systems faced with balancing the competitive priorities of patient-care responsibilities and educational needs. To evaluated the effects of a major organizational change on the inpatient service of an internal medicine residency program in a vertically integrated health system. We changed the structure of our program from a system in which the hospitalized patients' primary physicians were responsible for daily inpatient management, while teaching was assigned to a defined teaching rounder, to a method in which the rounding attending was responsible for both teaching and patient care. Measurements before and after the change in the rounding system included: the McGill University clinical tutor evaluations, time-motion observations of house staff, patient satisfaction surveys, average length of stay data, and physician focus groups to assess physician satisfaction. The rounding attendings consistently received excellent to superior ratings by the house staff both before and after the implemented change. Compared to time-motion observations performed before the change, observations recorded after the change suggested that a greater percent of house staff time was spent on educational activities. The responses of patient satisfaction surveys indicated that the perception of quality of care remained high after the system change. Lastly, the average length of stay for patients on the general internal medicine and subspecialty services was reduced from 7.6 days before the change to 6.6 days after the change, a difference of 0.92 day (95% confidence interval 1.3 to 0.6, P < 0.001). Through organizational restructuring, it is possible to improve the quality of patient care and improving the efficiency of patient-care management.

Empirical evaluation of the transfusion medicine tutor

Abstract Previous research during the development of Antibody IDentification Assistant (AIDA) revealed that many medical technology students and other laboratory personnel have serious difficulties in determining the specificity of blood group alloantibodies, especially weak or multiple antibodies. Based on these previous results, AIDA was modified to provide a teaching environment for medical technology students. We report the results of a rigorous, objective evaluation of the resultant system, the Transfusion Medicine Tutor (TMT). The results show that the students who were taught by an instructor using TMT to provide the instructional environment went from 0 percent correct on a pretest case to 87 percent correct on posttests (n = 15). This increase compares with an improvement rate of 20 percent by a control group (n = 15) who used a passive version of the system with the tutoring functions turned off.

An Examination of a Tutor Evaluation Form for Problem-based Learning Curricula in Physical Therapy and Occupational Therapy Education

Log in or Register Subscribe to journalSubscribe Get new issue alertsGet alerts Enter your Email address: Wolters Kluwer Health may email you for journal alerts and information, but is committed to maintaining your privacy and will not share your personal information without your express consent. For more information, please refer to our Privacy Policy. Subscribe to eTOC Secondary Logo Journal Logo All Articles Images Videos Podcasts Blogs Advanced Search Toggle navigation Subscribe Register Login Articles & Issues Current IssuePrevious IssuesPublished Online First CollectionsFor Authors Submit a ManuscriptInstructions for AuthorsLanguage Editing ServicesAuthor Permissions For ReviewersJournal Info About the JournalEditorial BoardAbout the AcademyAdvertisingSubscription ServicesReprintsOpen AccessRights and Permissions All Articles Images Videos Podcasts Blogs Advanced Search

The structure of a combined medicine and surgery clerkship

The third-year combined medicine and surgery clerkship at the University of Kentucky College of Medicine is a student-centred, multidisciplinary course designed to foster learning skills and emphasize the teaching of principles rather than of isolated facts. The clerkship's curricular and educational objectives are based on a set of clinical subject areas designed to educate a third-year student as a generalist physician. Clerkship activities include clinical duties and four types of integrated activities: small-group problem-based learning sessions, clinical skills workshops, expert resource sessions and integrated reading assignments. Students are evaluated by department-based measures, problem-based learning tutor evaluations, and integrated, monthly quizzes. The clerkship itself is evaluated and monitored regularly by both formal and informal data collection.

A rating scale for tutor evaluation in a problem-based curriculum: validity and reliability.

An instrument has been developed to assess tutor performance in problem-based tutorial groups. This tutor evaluation questionnaire consists of 13 statements reflecting the tutor's behaviour. The statements are based on a description of the tasks set for the tutor. This study reports results on the validity and reliability of the instrument. Confirmatory factor analysis showed that a three-factor model fitted the data reasonably well. The three factors are: (1) guiding students through the learning process, (2) content knowledge input, and (3) commitment to the group's learning. Generalizability studies indicated that the rating scales provide reliable information with student responses of existing tutorial group sizes. It is concluded that the tutor evaluation questionnaire is a fairly valid and reliable instrument that can be used in staff development programmes.

Knowledge gain in a problem-based surgery clerkship.

To determine the magnitude of students' knowledge gain in a problem-based surgery clerkship and to identify the relationship of this gain to measures of clinical performance. Third-year students in a problem-based surgery clerkship at the University of Kentucky College of Medicine in 1990-91 were evaluated by a comprehensive set of objective measures: surgery subject examinations of the National Board of Medical Examiners (NBME) given as pretest and posttest to assess knowledge gain, two multiple-choice quizzes, a modified-essay examination, a standardized-patient examination, and an objective structured clinical examination (OSCE). The students were also evaluated by faculty tutors and preceptors and by their peers. NBME data were available for 66 students, and data were available on the other measures for 42 students. Statistical analysis involved two-way analysis of variance, single group t-test, Pearson correlations, and partial correlations. The students' knowledge gain was statistically significant. The posttest mean score did not differ significantly from the national candidate mean of 500. All but two of the other knowledge and performance measures (the preceptor and tutor evaluations) correlated significantly with the knowledge gain score. The highest correlations were for peer evaluations, the standardized-patient examination, and the modified-essay examination. The overall reliability of the eight measures of student knowledge and performance was .81; deleting any measure, except the preceptor evaluation, lowered the reliability below the benchmark of .80. The results suggest that a highly significant knowledge gain occurred during the problem-based clerkship and that this gain in knowledge was closely related to improved clinical performance.

A problem-based course in dental implantology

A problem-based predoctoral implantology course was implemented for ten senior dental students at the University of Texas Health Science Center at San Antonio. Students met in small tutorial groups three times a week to analyze patient cases developed for the course. As students explored the patient9s problems, they identified areas where their knowledge was deficient, as well as special interests, and prioritized these areas, known as learning issues, into a learning agenda for subsequent study. During unprogrammed free time between tutorials, students researched the learning issues, and came to the next tutorial prepared to discuss what they had learned and in so doing, helped their peers learn the material. The faculty tutor used non-directive questioning to facilitate group problem-solving and promote interaction but did not lecture or give answers. The cases were developed in the progressive disclosure paperback format, reflecting the evolving nature of the patient9s implant therapy. In conjunction with the PBL tutorials, students performed implant surgery and prosthodontics on a minimum of three patients and provided maintenance therapy on three to four other implant patients. Course evaluation consisted of three elements: (1) weekly peer evaluation by the students using a six item checklist, (2) tutor evaluation, using the same checklist, and (3) an oral examination. The tutor met individually with each student midway through the course and at the conclusion to summarize the peer evaluations and discuss the student9s performance, including contributions to group process. Student reaction to this PBL experience was enthusiastically supportive, and student performance exceeded faculty expectations.

Assessing the problem-based clerkship: The rationale for and costs of a multidimensional assessment program

The assessment program of the University of Kentucky Surgical Clerkship has been developed to measure not only knowledge acquisition but also the acquisition of problem-solving abilities and clinical skills. Student performance is evaluated both subjectively and objectively using the following methods: tutor, preceptor, and peer evaluations; multiple-choice examinations; computerized patient management problems; standardized patient examinations; objective structured clinical examinations; and surgery shelf examinations (NBME-II surgery). Such a multifaceted evaluation program requires a considerable investment of time and financial resources; estimates of both are included in this report. Because such a program more reliably assesses student performance, however, the costs are well justified.

Tutor evaluation of problem-based learning in behavioral sciences courses

To date, the formal evaluation of problem‐based learning (PBL) has relied more heavily on student opinions than on faculty opinions. The present study was designed to help correct this imbalance by surveying the faculty and students of three social and behavioral sciences courses that included case‐based PBL segments. Respondents rated the experimental PBL segments in comparison with the regular‐course discussion‐group sessions. The numerical ratings and qualitative comments suggested that the faculty and students were significantly positive about the PBL segments in comparison with those parts of the course that were taught by more traditional methods and that this was because the PBL format increased student motivation and helped focus discussions. The data further suggested that the faculty and student ratings tended to parallel each other and that the faculty, compared to the students, were significantly more positive about the PBL segments. Both the ratings and the qualitative comments suggested that...

A problem‐based course in dental implantology

A problem-based predoctoral implantology course was implemented for ten senior dental students at the University of Texas Health Science Center at San Antonio. Students met in small tutorial groups three times a week to analyze patient cases developed for the course. As students explored the patient's problems, they identified areas where their knowledge was deficient, as well as special interests, and prioritized these areas, known as learning issues, into a learning agenda for subsequent study. During unprogrammed free time between tutorials, students researched the learning issues, and came to the next tutorial prepared to discuss what they had learned and in so doing, helped their peers learn the material. The faculty tutor used non-directive questioning to facilitate group problem-solving and promote interaction but did not lecture or give answers. The cases were developed in the "progressive disclosure paperback" format, reflecting the evolving nature of the patient's implant therapy. In conjunction with the PBL tutorials, students performed implant surgery and prosthodontics on a minimum of three patients and provided maintenance therapy on three to four other implant patients. Course evaluation consisted of three elements: (1) weekly peer evaluation by the students using a six item checklist, (2) tutor evaluation, using the same checklist, and (3) an oral examination. The tutor met individually with each student midway through the course and at the conclusion to summarize the peer evaluations and discuss the student's performance, including contributions to group process. Student reaction to this PBL experience was enthusiastically supportive, and student performance exceeded faculty expectations.