Analysing Maltese Biology Examination Questions according to Cognitive Complexity

Abstract

This paper presents an analysis of Maltese Advanced Biology examination comprehension questions according to cognitive complexity. The research data consisted of 239 questions from 20 Summer examinations: 10 National and 10 at a public post-secondary Institution between 2010 and 2019. In this research, a qualitative approach and theory-driven content analysis method using Bloom’s Revised Taxonomy of Cognitive Objectives were employed. The 121 Institution and 118 National examination questions were placed in two categories: higher-order cognitive skills (HOCS) comprising Remembering, Understanding and Applying and lower-order cognitive skills (LOCS) comprising Analysing, Evaluating and Creating on the basis of earlier research. Data was given with tables as percentage. This research was guided by the following two questions: What kinds of cognitive skills and knowledge do Maltese Advanced Biology comprehension examination questions require? What is the proportion of marks being awarded to the different cognitive levels? In the National examination questions were allocated to five of the cognitive categories while less, four, in the Institution. Over a 10-year period, the majority of the questions (91.6% in Institution; 81.6% in National) required LOCS. In both types of examination, the highest percentage of questions were in the Remembering objective (53% in Institution; 48% in National), followed by Understanding (31.84% in Institution; 22% in National) and finally Applying (6% in Institution; 11% in National). The study highlighted that the Analysing objective was absent in every comprehension and the Creating category was represented by a mere1% in the National examination. The investigation was extended to determine the marks allocated to the different cognitive levels. The majority of the marks, (92.0% in Institution; 81.7% in National examinations) belonged to the LOCS, being allocated mostly in the Remembering and Understanding objectives. The research indicates that the examinations were overall, not cognitively demanding, but the National one was more intellectually challenging for a number of reasons, including a larger percentage of questions and marks categorised as HOCS. The Bloom’s Revised Taxonomy of Cognitive Objectives as used in this research gives a useful way for designing or analysing biology summative assessment tools. All three higher-order cognitive skills categories should be more evenly presented in future biology examinations.