Rule Space Model Research Articles

PROFICIENCY SCALING BASED ON CONDITIONAL PROBABILITY FUNCTIONS FOR ATTRIBUTES

ABSTRACTThis study introduces procedures for constructing a proficiency scale for a large‐scale test by applying Tatsuoka's Rule Space Model. The SAT Mathematics (SAT M), Section 2, is used for illustrating the process and the results. A task analysis is summarized in a mapping sentence, and then 14 processes and content attributes are identified for explaining the underlying cognitive aspects of the examinees' performance on the SAT M. Analysis results show that almost 98% of 2334 examinees are successfully classified into one of 468 cognitive states. The cognitive states are characterized by mastery or non‐mastery of the 14 attributes. Attribute Characteristic Curves, which are conditional probability functions defined on the SAT Scale, are introduced and used for interpreting an examinees' proficiency. Prototypes of a student's performance report and a group performance report are given as examples of possible ways for summarizing the analysis results.

EXTENDING THE RULE SPACE MODEL TO A SEMANTICALLY‐RICH DOMAIN: DIAGNOSTIC ASSESSMENT IN ARCHITECTURE

ABSTRACTThis paper presents a technique for applying the Rule Space model of cognitive diagnosis (Tatsuoka, 1983) to assessment in a semantically‐rich domain. Responses to 22 architecture test items, developed to assess a range of architectural knowledge, were analyzed using Rule Space. Verbal protocol analyses guided the construction of a model of examinee performance, consisting of processes for constructing an initial representation of an item (labeled understand), forming goals and performing actions based on those goals (solve), and determining whether goals have been attempted and satisfied (check). Item attributes, derived from these processes, formed the basis for diagnosis. Our technique extends Rule Space's applicability by defining attributes in terms of item characteristics and the causal relations between characteristics and the problem‐solving model.Data were collected from 122 architects of various ability levels (students, architecture interns, and professional architects). Rule Space successfully classified approximately 65%, 90%, and 40% of examinees based, respectively, on attributes associated with the understand, solve, and check processes of the problem‐solving model. The findings support the effectiveness of Rule Space in a complex domain and suggest directions for developing new architecture items by using attributes particularly effective at distinguishing among examinees of different ability levels.

Diagnosing Knowledge States in Algebra Using the Rule-Space Model

This paper illustrates the use of rule space as a tool to support cognitive analyses of students' mathematical behavior. The rule-space approach is explained and is then used to classify students into one of two methods for solving linear algebraic equations in one unknown and to diagnose their knowledge states in this topic. A 32-item test with open-ended questions was administered to 231 eighth and ninth graders. Two outcomes of the rule-space model are presented: (a) a classification of examinees into knowledge states resulting from the two solution approaches at the group level along with individual examples and (b) tree diagrams of the transitional relationships among the states for each strategy. Implications for using the feedback provided by the rule-space model in the context of instruction and assessment are discussed.

Diagnosing Knowledge States in Algebra Using the Rule-Space Model

Diagnosing Knowledge States in Algebra Using the Rule-Space Model

Applying an IRT-Based Cognitive Diagnostic Model to Diagnose Students' Knowledge States in Multiplication and Division with Exponents

The study applied a psychometric model-the rule-space model-to diagnose students' states of knowledge about how the exponents behave in multiplication and division of quantities with exponents. A 38-item test was administered to 431 Grade 10 students. Each item was characterized by a list of task attributes required for answering the item correctly, and each student was classified, based on his or her item-score pattern, into the most likely knowledge, state (i.e., attribute-mastery pattern) corresponding to an ideal item-score pattern. The following outcomes of the rule-space model were presented: (a) the results of the classification of examinees to knowledge states at the group level along with individual examples, (b) the mastery level of the underlying task attributes as evaluated at three different test-score groups, and (c) a tree diagram of the transitional relationships among the knowledge states that can guide the design of effective remediation. Implications for utilizing the feedback, provided by the rule-space model in the context of instruction and assessment are discussed.

DIAGNOSING KNOWLEDGE STATES IN ALGEBRA USING THE RULE SPACE MODEL

ABSTRACTThis paper illustrates the use of rule space as a tool to support cognitive analyses of students' mathematical behavior. The rule space approach is explained and is then used to classify students into one of two methods for solving linear algebraic equations in one unknown and to diagnose their knowledge states in this topic. A 32‐item test with open‐ended questions was administered to 231 eighth and ninth graders. The following outcomes of the rule space model were presented: (a) a classification of examinees into knowledge states resulting from the two solution approaches at the group level along with individual examples; (b) tree‐diagrams of the transitional relationships among the states for each strategy. Implications for using the feedback provided by the rule space model in the context of instruction and assessment are discussed.

A PSYCHOMETRICALLY SOUND COGNITIVE DIAGNOSTIC MODEL: EFFECT OF REMEDIATION AS EMPIRICAL VALIDITY

Abstract : The purpose of this study was to validate the results of cognitive diagnoses using the rule-space model and to demonstrate the usefulness of cognitive diagnoses for instruction. The results of the study strongly indicated that the rule-space model can effectively diagnose students' knowledge states and can point out ways for remediating their errors quickly with minimum effort. It was also found that the designing of instructional units for remediation can be effectively guided by the rule-space model, because the determination of all possible ideal item-score patterns, given an incidence matrix, is based on a tree structure of cognitive attributes, knowledge states, and items.

Bug distribution and statistical pattern classification

A model (called the rule space model) which permits measuring cognitive skill acquisition, diagnosing cognitive errors, detecting the weaknesses and strengths of knowledge possessed by individuals was introduced earlier. This study further discusses the theoretical foundation of the model by introducing “bug distribution” and hypothesis testing (Bayes' decision rules for minimum errors) for classifying subjects into their most plausible latent state of knowledge. The model is illustrated with the domain of fraction arithmetic and compared with the results obtained from a conventional artificial intelligence approach.