Lessons Learned from Using Competency Based Assessment (CBA) in a First Year Engineering Statics Course

Abstract

During the Fall term of 2020, the first year Statics course in the College of Engineering at theUniversity of Saskatchewan was taught remotely, and synchronously, using a competency-based assessment (CBA) implementation. CBA is commonly used in other professions, especially medical education and teacher education, but it has yet to see widespread use in engineering education.CBA can involve a number of key differences from conventional assessment practices. In this pilot, it involved the following key differences. Whereas previous versions of the course had involved assignments, labs, a midterm, and a final exam, each worth a certain weight in the overallcourse grade, the 2020 CBA version was broken into three modules, each with a “module test”. The module tests were superficially similar to a midterm, and there was no cumulative final exam. Open book assignments, quizzes, labs, and module tests consisted of questions and exercises that addressed a variety of learning outcomes (LOs) within the modules. The LOs were assigned weights in the overall course grade, as opposed to assigning weights to assessments themselves as in a conventional assessment system. Students could therefore overcome poor performances in early assessments of LOs, as better later results on the same LOs would replace the earlier results.A key feature of this approach was that students had at least two and typically three or more opportunities to exhibit competence with respect to the course’s LOs. Another key aspect of this CBA implementation was the division of course material into three levels or “types”. Type A materials were the most basic building blocks of the course i.e. basic definitions, calculations, and concepts. Type B materials were basic integrative problems e.g. solving a basic 2D or 3D particle or rigid body equilibrium question, or solving a basic truss. Type C materials were advanced or “tricky” integrative problems that probed deeper understanding and required more adaptive problem solving. Students were required to meet competency thresholds for Type A and B materials i.e. theyneeded to exhibit a minimal level of competency in the LOs in order to pass the course.Overall, the class excelled in this assessment format and anecdotal evidence suggests that the students enjoyed it. A summary description of the complete system will be presented in this paper, including how grades were determined, how assessments were conducted and evaluated, how LOs were determined, and how the three levels of material were arranged. As well, basic statisticalresults from the class’ performance will be presented, along with a number of observations made by the instructors and some anecdotal impressions conveyed to them by students. The observed outcomes will be compared with the CBA literature for related STEM contexts, although the remote learning/COVID context did obscure the causes and origins of some of the observed outcomes. Changes that will be made in next year’s implementation of the course will also be discussed.