A Practical Procedure to Build a Knowledge Structure

Abstract

Computerized assessment of knowledge is one of the most promising applications of knowledge space theory. The first requirement of any such assessment is to produce as accurate a picture as possible of the organization of the knowledge. So far, all procedures designed for this purpose rely exclusively on the query of an expert. Several experiments have shown the limitations of that approach in realistic conditions. One source of difficulty is the very high sensitivity of the querying algorithms to an expert's mistakes. Another source of difficulty concerns the validity of the expert: His or her knowledge structure may diverge greatly from the knowledge structure of the actual population. To solve these difficulties, the present paper proposes and simulates a two-step procedure. The first step implements a modification of an existing querying procedure. The modification implements an error-handling mechanism which lowers the incidence of an expert's careless errors. The second step consists in a refinement mechanism which relies on the knowledge assessments of many subjects to refine the very structure used by these assessments. For each step, it is shown that the underlying knowledge structure can be recovered.