Rule Space Model Research Articles

The Application of Big Data Technology in English Teaching Mode Innovation and Effect Analysis

Abstract Big data technology provides a detailed development of English language teaching, which is targeted through the assessment of cognitive level. In this paper, ConvSLSTM is used to describe the problem of learners’ knowledge level and build a cognitive framework. With the support of the framework, “recall rate” was introduced to quantify the English vocabulary test results, PSTM ability was adopted to reflect the acquisition of syntactic variants, and attention control was borrowed to evaluate pronunciation characteristics. Based on the initial quantification mentioned above, a hierarchical-level calculation was accomplished by combining fuzzy logic. Meanwhile, based on the estimation of different sides, a rule space model was created, and some corrections were made to achieve the cognitive diagnosis of the group level. In terms of the syntactic developmental trajectory of the girl learners, the maximum value of the range of syntactic variation of the subjects appeared in the middle and late stages. During the early stages of development, the variation range is between 10-35, and the bandwidth is 25, with very little variation. However, in the middle stage of development, it basically starts to oscillate significantly between 10-85, at which time the bandwidth increases to 75. Big data technology has created a comprehensive measurement framework for learners’ English proficiency.

Development of a New Learning Progression Verification Method based on the Hierarchical Diagnostic Classification Model: Taking Grade 5 Students’ Fractional Operations as an Example

AbstractLearning progressions can reflect students’ continuous in‐depth thinking development paths, and their establishment is an iterative process from the construction of hypothetical learning progressions to the verification of that hypotheses. Considering the limitations of the existing verification method of learning progressions based on a rule space model, this study put forward a new validating method based on the hierarchical diagnostic classification model (HDCM) and developed a learning progression with grade 5 students’ fraction operations as an example. An expert group first abstracted the attributes and then developed a hypothesized learning progression. Next, the HDCM was used to analyze the test data collected from 817 fifth‐grade students, followed by the revision of the learning progression hypothesis. Results showed that (1) the fractional operations involved five attributes: basic operation (A1), reduction of a fraction (A2), changing fractions to a common denominator (A3), split with mix number (A4), and borrowing (A5); (2) the constructed learning progression featured four levels: Level 1 with A1, Level 2 with both A1 and A2, Level 3 with the first four attributes, and Level 4 with all five attributes.

Cognitive Learning Assessment Based on FAHP and RSM: A Case Study of Introduction to Network course

With the progress and innovation of digital technology, people's knowledge acquisition, information exchange, and communication modes have changed significantly. The universality of emerging technology media enables instructional designers to use digital media to provide different teaching materials and teaching methods to meet the needs of individual learners according to their differences. Accordingly, this study takes the Introduction to Network course as an example, proposes a Fuzzy Analytic Hierarchy Process combined with Rule-Space Model to guide and analyze students' learning status, and imports Microsoft Technology Association international certification as empirical analysis. The study revealed that, the experts believe that “understanding protocols and services” is the most important part of the basic knowledge of the Internet, followed by “understanding network infrastructure” and “understanding network hardware.” In the section on learning performance, the average progress rates of two class were 69.44% and 62.20%, respectively, the students' scores were significantly improved. In the section on obtaining Microsoft Technology Association certification, the passing rates were 93.44% and 93.10%, which were significant improvements in certification passing rate. In the future, this study can be applied to other fields to improve the relative learning effect and make it broader and more diversified.

Developing a Learning Progression of Buoyancy to Model Conceptual Change: A Latent Class and Rule Space Model Analysis

We applied latent class analysis and the rule space model to verify the cumulative characteristic of conceptual change by developing a learning progression for buoyancy. For this study, we first abstracted seven attributes of buoyancy and then developed a hypothesized learning progression for buoyancy. A 14-item buoyancy instrument was administered to 1089 8th grade students to verify and refine the learning progression. The results suggest four levels of progression during conceptual change when 8th grade students understand buoyancy. Students at level 0 can only master Density. When students progress to level 1, they can grasp Direction, Identification, Submerged volume, and Relative density on the basis of the prior level. Then, students gradually master Archimedes’ theory as they reach level 2. The most advanced students can further grasp Relation with motion and arrive at level 3. In addition, this four-level learning progression can be accounted for by the Qualitative–Quantitative–Integrative explanatory model.

Rutas de mejoramiento e informes de diagnóstico cognitivo con el métodorule space

Given that the development of models of cognitive diagnosis has expanded in recent years, there is increasingly a need to communicate the results to different publics in an effective way. Hence, this article sets forth the use of the rule space model in the cognitive diagnosis of structures of knowledge and skills in mathematics, shown in a representative sample of students in Costa Rica who took the National High School Diploma Test in that subject. With the aim of presenting the analyses pertaining to the chosen cognitive psychometric method, the article reports on empirical evidence for the validity of the test, as well as the formulation of the incidence matrix and the classification rate of the students. It concludes that people with the same scores in the test show differences in their probabilities in the mastery of attributes, which is why there is a need for both individual and collective reports which take the differences in the mastery of cognitive attributes into account, so they can be employed to create strategies focused on the strengthening of skills. And likewise, ensure that the information of a diagnostic nature which is obtained from such models is comprehensible and easy to interpret in terms of descriptions of cognitive attributes, in order to trace out learning routes, according to the probabilities of the mastery of skills which reveal the strengths and weaknesses of the student body, and in turn, come up with remedial educational measures.

EDUCATIONAL DATA MINING APPROACH FOR ENGINEERING GRADUATE ATTRIBUTES ANALYSIS

Curriculum improvement and graduate attributes assessments have become recently a serious issue for many Canadian engineering schools. Collecting assessment data concerning graduate attributes and the students’ learning is an important step of curriculum evaluation and the continuous improvement process. To be successful, this improvement process needs appropriate methods and tools for data analysis.Recent developments in the field of Psychometrics and Educational Data Mining (EDM) provide multidimensional item response models able to take into account student and curriculum attributes as parameters. The primary intent of these new models is to predict student successes based on students past performance and the assessment map underlying the tests they completed.We demonstrate in this paper that these models can also be used to analyze the assessment map. In the psychometric and Educational Data mining literature, assessment maps are usually represented as a parameter that associates items to competencies in a matrix called Q-matrix. This concept draws its origins from the Rule-Space Model that was introduced in the eighties to statistically classify student item responses into a set of ideal response patterns associated to different cognitive skills.A method based on the Additive Factor Model has been successfully implemented to analyse the Q-matrix corresponding to the assessment maps used in the graduate assessment process. The results of 17 volunteering anonymous students completing 36 courses at the Université de Moncton between winter 2010 and fall 2015 semesters was analysed with our method. Results obtained provided interesting and useful information regarding the assessment map and the overall assessment process that are presented and discussed in this paper.

Applying the Rule Space Model to Develop a Learning Progression for Thermochemistry

We used the Rule Space Model, a cognitive diagnostic model, to measure the learning progression for thermochemistry for senior high school students. We extracted five attributes and proposed their hierarchical relationships to model the construct of thermochemistry at four levels using a hypothesized learning progression. For this study, we developed 24 test items addressing the attributes of exothermic and endothermic reactions, chemical bonds and heat quantity change, reaction heat and enthalpy, thermochemical equations, and Hess’s law. The test was administered to a sample base of 694 senior high school students taught in 3 schools across 2 cities. Results based on the Rule Space Model analysis indicated that (1) the test items developed by the Rule Space Model were of high psychometric quality for good analysis of difficulties, discriminations, reliabilities, and validities; (2) the Rule Space Model analysis classified the students into seven different attribute mastery patterns; and (3) the initial hypothesized learning progression was modified by the attribute mastery patterns and the learning paths to be more precise and detailed.

Developing a learning progression for number sense based on the rule space model in China

The current study focuses on developing the learning progression of number sense for primary school students, and it applies a cognitive diagnostic model, the rule space model, to data analysis. The rule space model analysis firstly extracted nine cognitive attributes and their hierarchy model from the analysis of previous research and the mathematics textbook used in Beijing. A cognitive diagnostic test for number sense was then developed based upon the cognitive attributes. Finally, the model was used to analyse a sample of 1207 Chinese primary school students’ observed item responses to identify their knowledge states and to validate and modify the hypothesised learning progression. The results showed that the test was of good psychometric quality, and that the hypothesised learning progression was generally validated. By applying the rule space model, the hypothesised learning progression was modified at each level. The results also showed that students in grade 3, grade 4 and grade 5 were mainly classified into level 1 and level 2, level 2–level 4 and level 5 of the modified learning progression, respectively. These results suggest the feasibility and benefits of using cognitive diagnostic models to develop learning progressions.

Application of a Cognitive Diagnostic Model to a High-Stakes Reading Comprehension Test

General cognitive diagnostic models (CDM) such as the generalized deterministic input, noisy, “and” gate (G-DINA) model are flexible in that they allow for both compensatory and noncompensatory relationships among the subskills within the same test. Most of the previous CDM applications in the literature have been add-ons to simulation studies. Although there are some applications of CDMs such as the Fusion Model and the Rule Space Model to educational assessment data in general and second-language data in particular, there are few studies applying general models such as the G-DINA. The purpose of the present study was to demonstrate the application of the G-DINA to the reading comprehension data of a high-stakes test. To this end, an initial Q-matrix was developed, validated, and cross-validated. The skill profiles of the test takers were estimated using the “CDM” package in R. Throughout, the process of constructing and validating a Q-matrix was elaborated on, the benefits of general models were emphasized, and implications for research investigating inter-skill relationships were discussed. Finally, suggestions for further research, to better take advantage of the flexibilities of general diagnostic models, were presented.

Recommendation system based on rule-space model of two-phase blue-red tree and optimized learning path with multimedia learning and cognitive assessment evaluation

Among the various indicators used by the Ministry of Education to assess the learning performance and competencies in Taiwan, a highly emphasized one for students in the vocational education system is the numbers of professional certification they have, which is also an important factor for vocational students to gain suitable job opportunities and to enhance their working competitiveness. As a result, the importance of obtaining professional certifications can never be over emphasized. Specifically, the numbers of certifications they obtained is highly related with the numbers of job opportunities they can expect. In this research, we propose a RS (Recommendation System) based solution. The proposed solution combines two-phase Blue-Red trees of Rule-Space Model and the optimized learning path, and is focused on remedying and analyzing the learning status of MTA courses with the goal of enhancing students’ pass rate of MTA certifications. In phase one, we then identify three SGs (Skill Groups) based on course information from the Certiport of Microsoft certification center, and the three SGs can be utilized for producing both concept maps and Blue-Red trees. In phase two, we classified ten chapters of MTA course into three SGs identified in phase one based on the similarities observed in the ten chapters and the three SGs. The three SGs will then be used for generating the needed concept maps and groups of Blue-Red trees. In this research, we generated three of each. The analysis is based on Rule-Space Mode for all learning objects in each skill group of phase two. For each pair of learning objects, we define the RW (Relation Weight) of them. From all learning paths, we calculate the Confidence Level values of each adjacent pairs of learning objects. Finally, we obtain the optimized learning path through the adoption of the inferred optimized learning path derivation algorithm from the combination of RW (Relation Weight) and CL (Confidence Level). It can be used in OCMLS (Online Course Multimedia Learning System) that recommended the optimized learning path of learning objects for learners to online self-learning, or to RS (Recommendation System) that provides the basis of self-learning remedies for RFRC (Recommended Form of Remedial Course). By adopting this recommendation system for giving guidance for students in preparing for the MTA (Microsoft Technology Associate) certification, we have observed good results in learning performance and pass rate.

Detection of Misconceptions and Misleading Questions by Using Quantitative Diagnostic Assessment

Researches into E-Learning pay emphasis on how the technology is applied for comprehensive learning support in various perspectives. Lots of advantages are involved while performing the E-Learning. For example, timely contents are applicable to be delivered via the Internet, and in a similar manner, instant feedback which identifies the learning performance would be obtained through an online assessment system. In this article, a diagnostic approach for learning performance evaluation of student is proposed. Two primary issues are concentrated based on the analysis of quantitative assessment results. First the authors identify the misconceptions on specific subject(s) that students have learned and provide remedial suggestions about their learning status to students. Second, because questions may have several incorrect information to students, hence the authors approach provide suggested revisions to instructors. The proposed approach employs the Rule-Space model and the Student-Problem chart and is implemented in an online assessment system to obtain corresponding data (e.g., learning behavior) and performance (e.g., learning status). The feasibility is demonstrated through an empirical study (i.e., a series of pre-post tests) which was conducted in a domain-specific course (i.e., Fundamental of Information Technology) with around 450 students in a calendar year.

The best learning order inference based on blue-red trees of rule-space model for social network

Network learning is becoming increasingly popular today. It is getting important to develop adaptive learning by social network that can be applied in intelligent e-learning systems, and provide learners with efficient learning paths and learning orders for learning objects. Therefore, we use the rule-space model to infer reasonable learning effects of blue-red trees and their definitions through analysing all learning objects of courses within system. We can also define all part learning of sub-binary trees from a course and derive all learning paths from each part learning of sub-binary tree based on the premise that we had inferred nine learning groups of social network grouping algorithms. Most importantly, we can define the relation weight of every learning object associated with the other learning objects, and separately calculate the confidence level values of between two adjacent learning objects from all learning paths. And finally, we can find the optimal learning orders among all learning paths from a sub-binary tree in the case of ITE course.

Comparison and Selection of Five Noncompensatory Cognitive Diagnosis Models Based on Attribute Hierarchy Structure

Attribute hierarchy structure(AHS),which was considered as the basis of cognitive diagnosis,could largely affect the classification accuracy.However,in practical work,it was very difficult to determine whether the specified AHS was rational or not.Thus,it is necessary to explore how the AHS will affect the classification accuracy.This paper investigated the effect of different AHSs on the accuracy of diagnosis.And two AHSs were under investigation,one is the correctly identified AHS and the other is the incorrectly identified AHS.The commonly used Monte Carlo simulation method was employed to generate the data.And five cognitive diagnostic models,Rule Space Model(RSM),Attribute Hierarchy Model(AHM),General Distance Decision(GDD) Model,DINA_HC model and DINA model,were used to fit the same data.The results indicated that:(1) When the AHS was correctly identified,the attribute match ratios(AMRs) under RSM and AHM were both relatively low,while the AMRs under GDD,DINA_HC and DINA models were all relatively high.Furthermore,the AMRs under DINA_HC and DINA models were larger than that of GDD model.(2) When the AHS was incorrectly identified,the AMRs under RSM,AHM and GDD models were all relatively smaller compared to the case in which AHS was correctly identified,which indicated that the AHS significantly affects the accuracy of diagnosis of the RSM,AHM and GDD models.On the other hand,the influence of AHS on the accuracy of the diagnosis of DINA_HC model was moderate.But the accuracy of the diagnosis of DINA model will not be influenced by the AHS because the AHS information was not used in the DINA model.

A Learning Path analysis of College C plus plus Programming based on Rule Space Model

A Learning Path analysis of College C plus plus Programming based on Rule Space Model

Application of Bayesian Networks to Identify Hierarchical Relation Among Attributes in Cognitive Diagnosis

It's very significant to correctly identify the hierarchical relation among attributes correctly when constructing a diagnosis test.As we know,there are various methods to identify the hierarchical relation among attributes in different cognitive diagnosis models(CDMs).Rule Space Model(RSM) is a kind of great influence CDM which was developed by Tatsuoka and her associates.In RSM,the task of attribute identification is performed after the test items have already been developed.And then an incidence Q matrix can be determined which reflects hierarchical relation.However,in Leighton,Gierl and Hunka's(2004) Attribute Hierarchy Method(AHM) the organization of attributes(attributes,number of attributes and attribute hierarchical relation) should be determined before developing the test items.In RSM and AHM the importance of correctly identifying the attributes and their hierarchical relation cannot be overstated,and the attributes and their hierarchical relations serve as the most important input variables to the models because they provide the basis for interpreting the results.The hierarchical relation among attributes describes the domain knowledge structures.Understanding the domain knowledge structures in highly specific detail provides a rational basis for proposing and evaluating potential improvements in the measurement of general proficiency.Otherwise,improvement remains largely a trial-and-error process.It has been proved that is not reliable to get the hierarchical relation by analyzing the test items,even the resulting hierarchical relations are not unique.Now,structure learning of Bayesian Networks would be introduced to study the hierarchical relation from the examinees' attribute mastery patterns.On the one hand,this paper conducted a simulation study in which Cui,Leighton and Zheng(2006)'s attribute hierarchical relation was employed,and the sample size of 5000 was used.In the simulated study,different sliping rates was considered for the purpose of testing the robustness of Bayesian structure learning.On the other hand,this paper conducted an empirical study to research the attributes which were commonly used in different numeration representation system converting,and the sample size was 189.Also the hierarchical relation generated by Bayesian structure learning and the hierarchical relation extracted by domain expert was compared.The result of simulation study and empirical study all showed the Bayesian Networks was a very useful tool in determining the hierarchical relation among attributes in the realm of education measurement and psychometrics.It is proved that the result for reference is valuable through structure learning of Bayesian Networks in simulated experiments and practical application.Meanwhile,the Bayesian Networks also can be used to test the hierarchical relation in RSM.

The Theory about CD-CAT Based on FCA and Its Application

Cognitive diagnosis (CD) plays an important role in intelligent tutoring system. Computerized adaptive testing (CAT) is adaptive, fair, and efficient, which is suitable to large-scale examination. Traditional cognitive diagnostic test needs quite large number of items, the efficient and tailored CAT could be a remedy for it, so the CAT with cognitive diagnosis (CD-CAT) is prospective. It is more benefit to the students who live in the developing area without rich source of teaching and distance education is adopted there. Although many researchers pay attention to CD, some flaws exist because it is still in its infancy (Leighton at el.2007) and only a few researches of CD-CAT are gotten. In Tatsuoka’s Q-matrix theory, it is wrong that the rows/columns could form a Boolean lattice. Based on Rule Space Model (RSM) and the Attribute Hierarchy Method (AHM), Formal Concept Analysis (FCA) is applied into CD-CAT. Concept lattices are served as the model of CD. The technology of item bank construction, item selection strategies in CD-CAT and estimation method are considered to design an initiatory and systemic CD-CAT, which diagnoses examinees on-line and offers remedial measure for examinees in time. The algorithms of constructing concept lattice for CAT, diagnosing examinees and offering the best remedial measure to examinees are discussed theoretically in detail. The result of Monte Carlo study shows that examinees’ knowledge states are well diagnosed and the precision in examinees’ abilities estimation is satisfied.

Modification of Tatsuoka’s <I>Q</I> Matrix Theory

In Tatsuoka's Rule Space Model(RSM)and in Attribute Hierarchy Method(AHM)(Leighton et al.,2004),attributes and hierarchy serve as the most important input variables to the model because they provide the basis for interpreting the results in this approach to psychometric modeling(Gierl,et al.,2000).The hierarchical relation among the attributes is represented by adjacency matrix.From the adjacency matrix,the reachability matrix could be derived,which may then play an important role for deriving the reduced Q matrix(Gierl,et al.,2000).The reduced Q matrix is used to derive the examinee's knowledge state vector,which is a core concept in the Rule Space Model.In this paper,some flaws of Tatsuoka's Q matrix theory(1991,1995)are discussed,and some remedies are proposed,especially through a series of new algorithms.These algorithms are useful in the Rule Space Model and in the Attribute Hierarchy Model to construct a Q matrix when the reachability matrix is given,and are useful to calculate the ideal/expected response patterns without using the Boolean Descriptive Function.These algorithms demonstrate two facts:firstly,the reachability matrix is the most important tool in constructing a cognitive test,and could help increase the diagnosis accuracy;secondly,use of these algorithms can remedy the flaws in the Tatsuoka's Q matrix theory.Furthermore,the new algorithms have other advantages,such as that they reduce computational burden for some complicated tasks requiring heavy numerical operations.Hence,the proposed methods in the paper may enrich the applications of the Q matrix theory.

An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS-R

This study illustrates how a diagnostic testing model can be used to make detailed comparisons between student populations participating in international assessments. The performance of Turkish students on the TIMSS-R mathematics test was reanalyzed with a diagnostic testing model called the Rule Space Model. First, mathematical and cognitive skills (‘attributes’) measured by the test were determined. One hundred sixty-two items were coded in terms of their attribute involvement, creating an incidence matrix—the Q-matrix. Using the Q-matrix and the student response data, each student’s attribute mastery profile was determined. Mean attribute mastery levels of Turkish students were computed and compared to those of their American peers. It was shown that Turkish students were weak in algebra and probability/statistics. They also demonstrated poor profiles in skills such as applying rules in algebra, approximation/estimation, solving open-ended problems, recognizing patterns and relationships, and quantitative reading.

Linkage between teacher quality, student achievement,and cognitive skills: A rule-space model

The topic of teacher credentials and student performance is revisited in an international setting using the TIMSS-99 data. The lack of consistent positive link between credentials and performance can be explained via three routes: measurement problem of “teacher quality” input, measurement problem of “student outcomes”, and the production function form that is assumed to link the input and the output. Although there is a small literature focusing on student outcome measurement problems, suggesting the use of cognitive achievements rather than test scores, in most cases those cognitive measures are nothing but math and science scores. This study contributes to the literature by borrowing from the measurement and psychometrics theories to decompose single scores into three categories of cognitive abilities. The hypothesis is that teachers may play a crucial role in the development of some student cognitive skills while not in the others. Using a “rule-space” model, this article identifies three cognitive skills: the process skill, the reading skill and the mathematical think skill. The study finds that: (1) in general teacher credentials have no effect on any type of cognitive skill development as well as on the test score, and (2) the within-teacher variance of student performance is much larger than between-teacher variance in Japan and Korea, whereas the reverse is true in the US and the Netherlands. The phenomenon of “private tutoring” is quoted as an explanation of this pattern.

An NCME Instructional Module on Exploring the Logic of Tatsuoka's Rule‐Space Model for Test Development and Analysis

K. Tatsuoka's rule‐space model is a statistical method for classifying examinees' test item responses into a set of attribute‐mastery patterns associated with different cognitive skills. A fundamental assumption in the model resides in the idea that test items may be described by specific cognitive skills called attributes which can include distinct procedures, skills, or processes possessed by an examinee. The rule‐space model functions by collecting and ordering information about the attributes required to solve test items and then statistically classifying examinees' test item responses into a set of attribute‐mastery patterns, each one associated with a unique cognitive blueprint. The logic of Tatsuoka's rule‐space model, as it applies to test development and analysis, is examined an this module. Controversies and unresolved issues are also presented and discussed.