Rubrics for assessment item difficulty in engineering courses

Abstract

A course in Engineering programs is designed and conducted to facilitate students to acquire a set of competencies to meet a subset of Program Outcomes(Graduate Attributes) identified by the accreditation agency. Formative and summative assessments, if in alignment with the competencies, enable the instructor to guide the students to learn well, and to measure the level of attainment of competencies. The difficulty levels of the assessment items are a measure of mastery of the competencies. Well defined difficulty rubrics provide guidelines to the instructor to prepare assessment items with stated difficulty levels. Several difficulty rubrics which are learner dependent or independent, subject specific or non-specific and score-based or score-independent to characterize the difficulty level are reported in the literature. This paper presents a set of learner-independent and score-independent difficulty rubrics. The parameters that characterize these rubrics are arranged in a hierarchy. The nodal parameters are content, task and structure (stimulus) of assessment items. The leaf and atomic parameters are identified in the framework of Anderson-Bloom taxonomy. The parameters for a given assessment item are identified and difficulty is determined by applying weight based and fuzzy logic rule based methods. The efficacy of the proposed item difficulty rubrics is validated by designing different difficulty level items for the course Basics of Digital Systems, and validating the difficulty of the designed items by the subject matter experts.