Computer Testing to Document Student Achievement of Learning Outcomes

Abstract

To the Editor. Every course should have published learning outcomes describing what students should be able to do upon successful completion of the course.1 Assessing how well the students have achieved the learning outcomes for the course is very important as this can provide evidence of learning for the student and a measure of the effectiveness of the course. Evidence of learning is important for accreditation purposes and can provide data that can be used to improve the course.2 Documenting student achievement of outcomes can be done in many ways such as having students complete an objective structured clinical evaluation (OSCE), paper, presentation, or project. Other common methods to provide evidence of learning are to give assessments in the form of quizzes and examinations. Each examination and quiz question within a course should link to the learning outcomes of the course; consequently students who answer questions appropriately have demonstrated some knowledge towards the achievement of the linked learning outcome. Analyzing these results will help identify how well the students demonstrated achievement of an outcome. The purpose of this letter is to share a process that documents student achievement of the course learning outcomes using computer-based testing. The examples provided are from a Top 200 course that I teach. The Top 200 course is a 1-credit course offered in the fall semester of pharmacy students’ first year. Students learn basic information about frequently prescribed medications such as brand and generic names and dosing for commonly prescribed medications. Each question on the test is linked to the appropriate Accreditation Council for Pharmacy Education (ACPE) Appendix B and/or D outcomes, the course learning outcomes and program learning outcomes.3 The instructor may link the question to other outcomes or learning domains as desired. The Top 200 course has 4 tests plus a final examination. Once the students complete a test the instructor can look at the results and see what percentage of students answered a question correctly. Because each question is linked to ACPE outcomes, course learning outcomes, and program learning outcomes, the instructor can also see how well the student performed on each learning outcome. Here is an example. One of the questions on a test might be, “what is the brand name for levofloxacin?” This question addresses course learning outcome 3, ACPE Appendix D 3.02 and program learning outcome 1b. For simplicity sake, let’s say there are 10 questions on a test that address course learning outcome 3, the brand-generic type question. The computer program will track the overall percentage of students who answered the questions correctly and transfer that information to the course learning outcomes being tracked. This information can be generated at any point in time, thus individual test data and course data can be generated. This data can provide feedback to the instructor to enhance the teaching and learning activities for low scoring outcomes. If all of the instructors in the school use similar coding schemes for the ACPE and program learning outcomes, data can be tracked across all of the courses offered. From the program perspective, all of these outcomes can be analyzed by semester or year as a curricular evaluation. The individual student can also track how well they are doing per course learning outcome. This data can be used to focus their studies and can aid in advising. A critical point is linking the question to an outcome. The data will not be valid if the linkage between learning outcome and a question is not correct. Consequently, care must be taken when connecting learning outcomes to individual questions. In fact, peer review to verify this linkage is suggested. Some courses may provide evidence of achievement of learning outcomes without traditional testing, for example, use of an OSCE. If outcomes are achieved outside of a testing format, this data must be added to the data obtained from the computer testing format in order to have a complete picture of how well students achieved the learning outcomes. In conclusion, computer-based testing can assist the instructor by providing evidence for student achievement of learning outcomes by course or across multiple courses. Creating a report assessing achievement of learning outcomes across all courses can inform the curricular revision process.