CEAB Graduate Attributes Research Articles

Program-Level Assessment

Undergraduate engineering students are customarily tested for their content knowledge in exams which usually involve long-answer calculation-based questions. Although most programs use exam grades as a proxy for student knowledge uptake and evidence for attainment of CEAB graduate attributes, there is no guarantee that students have understood the underlying fundamental concepts let alone long-time retention of their newly acquired knowledge. This paper discusses the development of a yearly assessment in the form of a concept inventory that is administered once at the beginning of the academic year (pre-test) and repeated in about a year (post-test). The analysis of the results showed a normalized gain of 20% in student knowledge. Further analysis revealed that there are no statistically significant differences between the male and female students suggesting an overall equitable learning experience by gender in our program. The normalized gain for each core course of the curriculum was also calculated which serve as curricular feedback for instructors identifying areas where improved instructional methods would provide benefits. Finally, a report is produced for each student to provide them with valuable feedback on how well they learned and retained the fundamental concepts they learned over the past academic year.

WHAT IS AN ENGINEER: STUDY DESCRIPTION AND CODEBOOOK DEVELOPMENT

There are still many questions to answer regarding the implementation and ongoing use of the CEAB graduate attributes (GAs) to guide engineering education content and student progression. How well doour students know and understand the twelve GAs? Which ones do they find most important? Do the knowledge and importance of the GAs vary over the course of students’ programs, or among institutions? Do students’ definitions of an engineer reflect the GAs? How do the definitions reflect students’ evolving understanding of engineering identity? Given the similarity of purpose, philosophy, and complementarity of questions and methods regarding the student perspective on engineering, the GAs, and their development, researchers from three institutions across Canada joined forces to conduct a national study. The overarching objectives of the study are to provide insight on how undergraduate students’ engineering identities develop through the course of their programs using the CEAB GAs as a normative framework, and enable meaningful comparisons of the GAs rankings and learning cultures at the three institutions. The objectives of this paper are to present an overview of the study development, and describe the methods used to develop the French and English codebooks for analysis of the qualitative data. We discuss the disambiguation of the codes and lexicon with particular attention to the concepts of professionalism and leadership and the emergence of three inductive codes: Engineering Work, Societal Improvement, and Personal Characteristics. We close the paper with a few words on future work.

DESIGN DAYS BOOT CAMP 2.0: IMPROVEMENTS AND CONNECTIONS TO CEAB GRADUATE ATTRIBUTES

Engineering design is a critical skill that all engineering students are expected to learn and is often the focus of final year capstone projects and first-year cornerstone projects. In the Systems Design Engineering Department at the University of Waterloo, engineering design is introduced to the students during an intense two-day Design Days Boot Camp. Design Days was originally conceived of and run in Fall 2016. The Fall 2018 version, Design Days 2.0, included substantial improvements focused on adding two additional design activities and a writing activity, strengthening the connection with first year content, and providing a greater variety of team experiences. The methods of achieving the nine intended learning outcomes of Design Days 2.0 are discussed and connected to CEAB graduate attributes. This demonstrates that meaningful learning can be achieved during a two-day boot camp that will starts students on the path towards professional engineering. Other departments are encouraged to use the presented intended learning outcomes, graduate attributes connections, and Design Days 2.0 descriptions as a template for their own design boot camp. Finally, Design Days 2.0 inspired ideas for further improvements including the incorporation of a software-focused design activity, adding budgetary constraints, and providing an opportunity for student reflection.

Computer-supported collaborative knowledge building using a sustainability assessment framework and Microsoft Teams

Ashby’s sustainability assessment method provides a systematic framework for dealing with complex, multidisciplinary problems. It explicitly addresses many desired graduate attributes, and is an excellent framework to provide scaffolding for computer-supported collaborative knowledge building. This paper describes experience using Ashby’s method in conjunction with the Microsoft Teams collaboration platform in a third-year group sustainability assessment project. Students found Teams to be effective and easy to use, and the resulting group reports demonstrated development and application of most CEAB graduate attributes, particularly professionalism, impact on society and the environment, and life-long learning.

Determining the Content Validity of a Biosystems Engineering Program

This study was designed as an exploratorycase study to determine the relative importance anddependencies of the CEAB graduate attributes asperceived by engineering stakeholders of the University ofManitoba. The findings were used to examine the contentvalidity of the Biosystems Engineering program. Theoverarching objective was to explore how well theemphasis on graduate attributes development in theFaculty of Engineering at the University of Manitobareflect the graduate attribute importance reported by keystakeholders. Findings showed that all stakeholdersranked Individual and Teamwork and CommunicationsSkills as the top engineering competencies, and all CEABgraduate attributes were perceived to between 6.1% and10.9% relatively important. This was in sharp contrast tothe Biosystems Engineering program, which is comprisedof approximately 50% of the graduate attribute, AKnowledge Base for Engineering. In this paper, themethods and findings in the determination of the contentvalidity of an engineering program are presented anddiscussed.

Utilizing the CDIO syllabus to reveal CEAB Graduate Attribute Pathways in a Mechanical Engineering Curriculum

Many Canadian engineering programs offercourses through a pre-requisite approach: coursework,and therefore knowledge, is assumed to build throughout aprogram. The pre-requisite approach allows postsecondaryinstitutions to monitor the pathway a studenttakes throughout a program, and is assumed to be informedby knowledge journeys. In our experience, however, prerequisitesmay only be loosely based on scaffoldeddevelopment of student learning. Course pathwayssometimes place a greater emphasis on administrativeconvenience and historical relationships, rather thanreflect an up-to-date or meaningful developmental journey.These “pathways” appear to be particularly tenuous whenit comes to professional skills development, despite anincreasing emphasis on their importance (see Canada’sGraduate Attributes 6-12 (Canadian EngineeringAccreditation Board, 2017)). The pre-requisite model mayreinforce a bureaucratic approach to professional skillsdevelopment, inhibiting flexibility and innovation in coursedelivery by allowing administration, rather than learningoutcomes to guide student learning

INVESTIGATING THE RELATIVE IMPORTANCE OF THE CEAB GRADUATE ATTRIBUTES: STUDY DESIGN AND INITIAL FINDINGS

Abstract - This study represents the PhD thesis research of the lead author. The greater study is designed as a mixed-methods explanatory case study. The overarching objective is to explore whether the CEAB graduate attributes are emphasized in the engineering programs in the Faculty of Engineering at the University of Manitoba in the most optimal way. The first phase of the study is designed to determine the relative importance and the levels of dependencies of the graduate attributes across three engineering stakeholder groups: faculty, students, and industry members. In this paper, the design and methodology of the first phase of this mixed methods research study are explained, and the preliminary findings from the participation rates and participants’ familiarity with the graduate attributes on the survey data are presented. Results show that a sizable percentage of students, faculty and industry members are not highly familiar with the CEAB graduate attributes. Therefore, work to develop a common knowledge about the CEAB graduate attributes needs to continue.

DESCRIBING AND MEASURING THE ENGINEERING KNOWLEDGE BASED USING CONCEPT DOMAINS

Abstract – We describe the chemical engineering knowledge base in terms of five distinct concept domains: i) mathematics and computation, ii) conservation, iii) equilibrium and spontaneity, iv) rates, and v) the structure and property of materials. These concept domains underpin the curriculum and evolve from disparate subject domains presented in the first year into a cohesive whole by graduation. The knowledge base for chemical engineering can thus be expressed in terms of achieving threshold concepts related to each of these domains. 
 This formulation of the knowledge base suggests that it may be examined using concept inventory testing. We provide examples of how such testing can be implemented in order to produce meaningful data on students’ level of concept attainment. We believe that this approach may be of interest to others as a robust and sustainable method for the ongoing assessment of CEAB Graduate Attribute 1.

Adapting CEAB Graduate Attributes to Promote Graduate Student Success in the Canadian Engineering Community

Abstract –Graduate engineering programs at Memorial University’s Faculty of Engineering and Applied Science are not accredited and as such there is no information available on non-technical professional skills development. These skills are essential for successful entry into the Canadian engineering community. To assess the current graduate program, an assessment methodology was developed based on the CEAB graduate attribute assessment process used for undergraduate program accreditation. An initial application of this methodology to Process Engineering indicates that all professional skills are not covered for all students. This lead to modifications for current programing that will ensure all students are exposed to the skills needed for a successful career in engineering.

Implementation of Service Learning into Engineering Design

This paper will discuss themes relatedto the implementation of the engineering designprocess by two second-year engineering studentswhile working in an international setting on a servicelearning project. In February 2015, the authors of thispaper designed, tested, and implemented a novelwheelchair attachment to improve the mobility ofpersons with disabilities in Kenya. This project wascarried out in its entirety during a period of 2 weeks,while staying in the small village of Mikinduri, locatedin Kenya’s Eastern Province. The scope of this paperwill include benefits of implementing such projectsinto engineering design curriculum, along withrecommendations based on the authors’ experiences.Topics such as CEAB Graduate Attributes covered,material availability, and communication barriers willbe compared and contrasted between standard andservice learning design projects

FIRST-YEAR VERSUS FOURTH-YEAR AND ONLINE VERSUS PAPER-AND-PENCIL RESPONSES TO A GRADUATE ATTRIBUTE SELF-ASSESSMENT TOOL AND EXIT SURVEY

A tool has been created that allows students to self-assess how well their program helped them achieve CEAB graduate attributes. These indirect self-assessments are used at the University of Waterloo in the Mechanicaland Mechatronics Engineering (MME) Department along with more direct measurements to inform analysis of programs. The tool uses an online survey platform to allow for randomized presentation of a subset of prompts related to the graduate attributes to each student in order to reduce the number of questions posed to any one student while ensuring coverage of all attributes.This tool has been deployed to first-year and fourth-year cohorts, and the differences in students’ self-assessments are discussed, in particular the increase in self-awareness that is expected to occur between first and fourth-year. The tool has also been deployed as part of an online exit survey administered to fourth-year students only, and similarities and differences in survey responses for online versus paper-and-pencil are discussed especially as they relate to free-form responses

THE LAC-MÉGANTIC TRAIN DISASTER AS A BASIS FOR A DESIGN PROJECT THAT EMPHASIZES SAFETY

Professional Engineers are ethically bound to regard public welfare as paramount, above all other considerations. Public welfare encompasses health, safety, economics, environmental and other considerations. These considerations are stressed throughout the CEAB graduate attributes. However, issues around public welfare are often absent from student learning experiences, particularly capstone design. This paper describes a capstone project motivated by the Lac- Mégantic train disaster. The students involved in this project researched the background of the accident, identified the contributing factors, and developed a problem definition to develop an improved braking system to prevent runaway trains. The proposed solution was a simple pneumatic drive connected to the handwheels of the manual brakes to reduce the time and effort involved in setting the brakes. This design could be retrofitted to existing rail cars at a cost of about $10,000 per car. Future improvements include implementation of a wireless control system to actuate and monitor the brakes from the locomotive cab.

RUBRICS AS A VEHICLE TO DEFINE THE TWELVE CEAB GRADUATE ATTRIBUTES, DETERMINE GRADUATE COMPETENCIES, AND DEVELOP A COMMON LANGUAGE FOR ENGINEERING STAKEHOLDERS

This paper discusses the evolution of a set ofrubrics for the 12 CEAB graduate attributes in theFaculty of Engineering at the University of Manitoba. Therubrics are intended as a pedagogical assessment tool forinstructors of individual courses as applicable, and forassessment at the program level. Individuals from faculty,industry and the University of Manitoba Centre for theAdvancement of Teaching and Learning have beeninvolved in the process of evaluating and revising boththe content and wording of the rubrics in order that theymeet the following criteria: (i) the foci and indicatorsadequately communicate the knowledge, skills, attitudes,values and behaviours that our engineering stakeholdersagree do define each attribute; (ii) the competency levelfor each indicator is representative of what engineeringeducators and stakeholders agree defines proficiency;and (iii) the language in the rubrics is consistent andagreeable to all engineering stakeholders. These rubricsare expected to accomplish a number of outcomes-basedpedagogical and accreditation goals, including: dividingthe attributes into teachable and measurable foci andindicators; defining competency levels; and becoming avehicle for the development of a common language forfaculty, students and industry when they discuss, teach,assess and acquire the knowledge, skills and behavioursof the CEAB graduate attributes. This paper reports onthe evolution of these rubrics, and outlines plans for theircontinued development and use within the faculty.

EXAMINING FOURTH YEAR MECHANICAL ENGINEERING STUDENT PERCEPTIONS OF GRADUATE ATTRIBUTE COMPETENCIES: YEAR THREE

This paper describes the third year of amixed methods explanatory case study to examine fourthyear engineering student perceptions of the CEABgraduate attributes in the Mechanical Engineeringprogram using a student exit survey and student focusgroup interviews. The purpose of this paper is to reportstudents’ perceptions of their graduate attributecompetency levels and the program’s graduate attributestrengths and weaknesses in a three-year continuum,supporting the findings with the data from three studentfocus group interviews. This investigation offers thefaculty an understanding of its student engineeringstakeholders’ perceptions of how the CEAB graduateattributes are manifest in the Mechanical Engineeringprogram. It will be used to provide feedback atinstructor, program and faculty levels as the Universityof Manitoba’s Faculty of Engineering furthers its effortstowards achieving a continuous cycle of improvement

ENGINEERING GRADUATE ATTRIBUTES EXHIBITED BY HIGH SCHOOL STUDENTS

Awareness and knowledge of both the engineering profession and engineering education programming is important for students in high school because strategic course choices must be made for students to qualify for university enrolment. This paper, a work-in-progress of a larger study, uses a qualitative analysis framed by the CEAB Graduate Attributes to gain insight into how teachers identify students who could become engineers. Participants clearly identified traits that describe the Knowledge Base attribute. Many participants identified other traits that described how students work; these did not fit easily within one attribute. The one attribute that was notably absent was Design.The findings describe that participants had a partial idea of the traits that would describe a potential engineer. This gap in knowledge supports work to develop a complete idea of the engineering profession at the high school level to ensure students can make informed course selection decisions and, in turn, career decisions

EXAMINING FOURTH YEAR ENGINEERING STUDENT PERCEPTIONS OF GRADUATE ATTRIBUTE COMPETENCIES: YEAR TWO

As the Engineering Faculty at the Universityof Manitoba shifts its curriculum from an input-based toan outcomes-based pedagogy, data from diverse sourcesare being collected. Among them, indirect data are beinggathered from students using a student exit survey. Thesurvey has been developed over the past three years toexplore graduating students’ perceptions of theiraptitudes and their engineering program’s strengths andweaknesses in regards to the 12 CEAB graduateattributes. It is comprised of the 12 attributes, with eachattribute further defined by six indicators. Theseindicators reflect the levels of Bloom’s Taxonomy ofEducational Objectives in the Cognitive Domain:knowledge, comprehension, application, analysis,synthesis and evaluation. The student exit survey was firstadministered to graduating mechanical engineeringstudents at the end of Fall semester 2012 and then at theend of Fall semester 2013. This paper describes thesecond year of the study, and discusses the datacomparatively with the findings from the first year. Thisstudy offers the Engineering Faculty an understanding oftheir Mechanical Engineering students’ experiences withand perceptions of the CEAB graduate attributes. It willbe used to provide feedback at instructor, program andfaculty levels as the University of Manitoba’s Faculty ofEngineering continues to implement its cycle of programdevelopment and improvement.

WHICH TYPE OF DESIGN PROJECT IS BEST: NARROW AND DETAILED OR BROAD AND CONCEPTUAL?

Engineering design is an essential part of the engineering curriculum, and it is important to select projects with appropriate scope and challenge to develop the desired skills subject to constraints on time, student ability and available resources. This paper considers two types of projects typically encountered in capstone design: detailed design projects, and conceptual design projects. Detailed design projects usually have a goal of constructing and testing a physical prototype, and the main focus is on CAD modeling, detailed analysis, engineering drawings, manufacturing processes, prototype fabrication, and testing. Conceptual design projects focus on the conceptual design stages, and typically do not result in a prototype. These projects are more open-ended, and focus on problem definition, background research, order-of-magnitude analysis, numerical simulation, technical and economic feasibility, and consideration of nontechnical aspects including impact on society and the environment. This paper compares and contrasts the two types of projects in terms of their characteristics, and evaluates them based on the CEAB Graduate Attributes. Both types of projects provide valuable and complementary design experience, and each type emphasizes different attributes.

Capstone Project Evaluation – Towards a Student-Centred Approach

Capstone projects present a particular challenge to assessment as compared to individual course work. Projects unfold over a year, culminating in a final report, begging the question of (balancing the) assessment of process over product. Projects across an individual department, let alone across the engineering faculty, can differ in content, the balance of depth versus breadth, the balance of research versus application, the balance of design versus implementation, and size of teams. One approach to managing this diversity is the use of broadly interpreted categories and no explicit weighted marking scheme, with the final grade determined by consensus in a department-wide meeting, held privately from the students. While flexibility and the authority of the supervising faculty member is acknowledged and maintained in this approach, the needs of the students are not necessarily best served in this approach. The movement toward learning outcomes, including the CEAB Graduate Attribute Criteria, is providing a well-understood and documented language for established indicators. This paper presents the results of an effort this year to incorporate CEAB graduate attributes into a system of marking rubrics. The goal is to better serve the needs of students with an assessment strategy that is based on explicit expectations and transparency, one which includes all deliverables, and yet still accommodates for diversity in project experiences. The paper will present the compound assessment instrument developed and used on select project teams, as well as the feedback of the students involved in the experience. The work done this year is seen as preliminary and the intent is to invite feedback to move towards broaden the adoption of the assessment instruments within our community.

Teaching Information Literacy Skills to Senior Undergraduate Engineering Students

Information Literacy Standards for Science and Engineering/Technology establish a set of abilities for students to recognize their information needs, access information effectively and efficiently, evaluate information critically, and use information in a legal and ethical way. The Standards aim to help students accomplish academic goals in their studies and build lifelong learning skills, and they are also well-aligned with the CEAB graduate attributes. Moreover, it is important that students acquire these skills before completing their undergraduate degrees since, upon entering the workforce, they will need to investigate problems and communicate complex ideas with colleagues and clients in both written and oral format, and they will not have the same support available as when they are in university. The authors, an engineering librarian and instructor, collaborated in the Fall 2012 semester to offer two information literacy training sessions within a final-year electrical and computer engineering course. This paper presents the topics and pedagogies used in this training and discusses the challenges that the authors encountered. Recommendations on integrating information literacy training into engineering curriculum are also provided.

THE ATTRIBUTE ASSESSMENT PROCESS AT THE UNIVERSITY OF MANITOBA: YEAR TWO

This paper describes the process in the second year of a three year study at the University of Manitoba that looks at how the 12 CEAB graduate attributes are manifested and measured in the engineering curriculum. The four attributes chosen for this year’s study were Problem Analysis, Use of Engineering Tools, Communication Skills, and Ethics and Equity. Nine instructors from each of the Departments of Biosystems, Civil, Electrical and Computer, and Mechanical Engineering were asked to consider the presence of these attributes in one of their engineering courses taught in Fall 2012. The checklist for this study was revised based on the results of the pilot study conducted in 2011-12, and in an effort to begin to define student attribute competency levels and demonstrate outcomes-based assessment. Similar to last year, this study found that the hard skills in engineering were assessed more frequently than the soft skills, and inparticular, there was little assessment evidence of Ethics and Equity. The majority of instructors reported using assignments and reports as evaluation tools, and communicating evaluations to students using numerical marks and written comments. Competency levels were defined in a variety of ways, highlighting the need to establish a common language for assessment. Finally, this paper reports on the challenges observed in the construction and administration of the survey and outlines next steps.