Evaluability assessment for engineering educators: Pre-conference workshop proposal and outline

Abstract

Listen

Translate article

Evaluability assessment (EA) is used to determine the readiness of a program for outcome evaluation. For engineering educators, obtaining meaningful outcome evaluation findings can be illusive. Determining whether a program is ready for such an evaluation, has critical importance, particularly as expectations for engineering education programs to have real impact, continue to increase. EA use is on the rise internationally with vibrancy in its application across a wide variety of disciplines and programs. Its use has expanded to include formative evaluation, implementation assessment, evaluation planning, program development, and technical assistance. EA is also being used to increase stakeholder involvement, understand program culture and context, and facilitate organizational learning and evaluation capacity building. However, there continues to be ambiguity and uncertainty about the method. Although several EA models exist, the essential elements of EA include stakeholder involvement, developing a program theory, gathering feedback on program theory, and using the EA. In this workshop, a modern model of EA will be presented that incorporates the essential elements of EA with current evaluation theory and practice. The components of this model include: (1) focusing the EA, (2) developing an initial program theory, (3) gathering feedback on program theory, and (4) and using the EA. Participants will learn the "What, Why, and How" of EA; specifically: What: Participants will learn the essential elements of EA and how they are incorporated in the EA model presented. Why: Participants will learn the important benefits and advantages of conducting an EA. How: Participants will learn how to implement the EA model presented. Participants will be exposed to a variety of case examples that illustrate features of EA that show how EA can be used across disciplines. Brief video clips of evaluators will be presented to illustrate how evaluators developed and carried out EA projects, issues that arose and how they were dealt with, and unique aspects that emerged in each EA. Participants will also engage in application exercises and related discussion to practice implementing the EA model. These exercises will come from actual engineering education projects and programs, clearly illustrating application of EA for engineering educators. We will administer a pre-workshop questionnaire to identify participant characteristics and prior experience with and interest in EA and program evaluation to better tailor the workshop to participant needs.