Advocating programme evaluation in new problem-based learning schools

Abstract

Context and setting A new system to monitor, evaluate and report on areas affecting the teaching and learning processes that accompanied curriculum change was developed through establishing an evaluations office at the Nelson R. Mandela School of Medicine (NRMSM). Why the idea was necessary The functioning of the new evaluations office was constraint by combining evaluations and implementation functions because of severe staff shortages during the implementation phase of problem-based learning (PBL). The purpose of this study was to investigate the efficacy of continuing a separate student feedback system through an evaluations office, as a means of providing direction for curriculum review as well as support for student learning and staff development activities at NRMSM. What was done To determine the usefulness of student feedback data, we conducted qualitative analysis of the feedback on 5 curriculum units in Year 2 (during 2004) and compared it with similar Year 2 student feedback (during 2005). The results were provided to student representatives, PBL facilitators and faculty members to determine the value of this evaluation system. The data were also made available to a panel of the Health Professions Council of South Africa (HPCSA) during an accreditation visit in 2005. Evaluation of results and impact After learning that student feedback had effected substantial changes in the Year 2 curriculum, students had an increased sense of their empowerment to partake in and influence curriculum decisions. They were grateful for the changes made that resulted in an improved teaching and learning climate. These included: a formally structured dissection programme; improved sequence of curricular learning units to avoid competition for textbooks; more structured feedback sessions after assessments; the inclusion of question and answer sessions at regular intervals to address issues before the conclusion of units; a noticeable increase in structured interactive large-group resource sessions; the availability of lecturers’ notes ⁄presentations, and the inclusion of subject study guides. Students were also appreciative of individual feedback they received based on facilitator reports, as were facilitators of their feedback from students. The 2005 student feedback revealed some remaining concerns focusing on the negative attitudes of some faculty members who did not allow discussion during lectures, did not make notes available, were not always audible to the class, or who cancelled classes without prior notification. Students provided suggestions to improve learning, including: more health care contacts and hospital visits during learning units; more specific objectives in PBL cases, and improved study guides in all curriculum units. The importance of the above data and the evaluations system was confirmed by the HPCSA’s recommendations. These included the separation of staff development and curriculum evaluation functions from that of curriculum implementation because of potential conflicts of interest. It is important that curriculum evaluation units demonstrate efficacy of their functions to all stakeholders, including students, faculty and administrators. The present study documented the utility of student feedback in improving the teaching and learning climate during the implementation of a new curriculum in a resource-constrained setting. Moreover, the importance of separating curriculum implementation from evaluation and staff development functions was confirmed.