The problem-solving potential of text evaluation

Abstract

Examination papers have until now received little attention in the field of document design. Although this theoretical framework and the domain of application may not be a perfect match, the principles of document design could have some heuristic value in the domain of assessment in education. The poor results of a group of South African technical college students from certain cultural and educational backgrounds merited an investigation into the diagnostic and remedial value of these principles. In this contribution examination papers are situated within a general model for problem-based text evaluation, representative of current thinking in document design. At a more specic level criteria for assessment in education are identied and validated as criteria for examination papers in particular. These criteria are mapped onto the textual elements of documents and represented in an evaluation matrix. The applicability of the matrix is demonstrated through the analysis of a selection of examples from national examination papers for career-oriented college subjects. It is suggested that any pretesting - text-focused, expert-focused or reader-focused - be preceded by thorough planning, taking cognizance of the macro-context of problem-based document design, valid micro-contextual criteria, and appropriate research methods. The outcomes should yield useful dividends at a practical as well as an academic level.