Are They Learning? Assurance of Learning by Curriculum Alignment to Professional Bodies of Knowledge

Abstract

Background – Measuring learning outcomes is a desirable activity in graduate programs. It is vital for programs that purport to educate practitioners in specific industries. Assessing that students adequately acquire prescribed knowledge and adjusting curricula upon discovering gaps is an important element in program evaluation and management. We present a process to align the program curriculum with industry standards to assure the acquisition of those desirable competencies. This paper presents how to: (1) select the appropriate professional BOK for each knowledge domain in question; (2) map a curriculum to the elements of the body of knowledge; (3) create assessment instruments based on the BOKs; (4) assess alignment; and (5) propose curricular revisions. Contribution – The process was applied to the curriculum of a technology management program master’s program to measure the student’s acquisition in three domains: data analytics, business analysis, and project management. Assessment instruments for each discipline based on corresponding BOK were developed and used to measure students’ acquisition of the skill sets. The results were used to drive curriculum reform. Assessment Questions - To align the curricula to professional BOKs, we asked: (1) whether students graduating from the program had acquired adequate competencies in each TM domain; and (2) If they had not, how the curriculum should be modified to help students acquire those competencies. Methods - The curriculum was reviewed, and we assured initially that topical coverage of the pertinent courses aligned adequately with each BOK knowledge domain. We ascertained curriculum alignment to the BOK by using text similarity scoring of the course descriptions to the BOK domain descriptions. Assessments in the form of multiple-choice exams based on each BOK were created and administered to students in their last semester of study. Results were analyzed, gaps were discovered, and curricular changes to identified courses to fill those gaps were proposed to faculty. Results - Using the existing curriculum, we found that most of the students at the end of the program were able to pass the assessments based on the BOKs. The exam results were sufficiently granular to allow proposals of modifications to the curriculum and course contents to improve the passing rate in future assessments. This appears to be an adequate process of identifying gaps in students’ knowledge of industry-specific topics.