Multidimensional Random Coefficients Multinomial Logit Research Articles

Domain-Linked and Domain-Specific Competence: a Validation Study of a Two-Dimensional Model of Economic Vocational Competence in Germany

Modeling vocational competence is increasingly crucial for monitoring and enhancing the quality of Vocational Educational Training (VET), particularly in the context of ongoing international comparative studies known as "large-scale assessments" of vocational education and training. This study endeavors to provide well-structured and guideline-compliant empirical evidence for the validation of the two-dimensional construct of economic vocational competence, advancing beyond the current state of research. A sample of 1438 first-year apprentices from two federal states in Germany participated as test-takers. The authentic assessment framework comprised 24 items, assessing two dimensions of vocational competence: domain-linked competence and domain-specific competence in the business/commercial domain. Measurement invariance was assessed across (1) federal states and (2) versions of test booklets, and the Multidimensional Random Coefficient Multinomial Logit model was employed to examine the quality of the two-dimensional vocational competence construct. The results supported the validity of the structure, highlighting the differentiation between domain-linked competence and domain-specific competence. This provides a more substantively accurate representation of trainees' vocational competence compared to a unidimensional model.

Developing and validating the construct maps to assess mathematical proficiencies

The study was aimed to develop and validate a measurement test to assess mathematical proficiencies in the numbers and algebra strand. A total of 125 seventh-grade learners with diverse learning abilities from secondary educational institutions under the management of the Khon Kaen Educational Service Area 25, Thailand was chosen as respondents. The researchers employed design-based research consisting of four building blocks to construct a prototype utilizing a construct modeling approach. Firstly, the researchers developed construct maps to identify the learners’ mathematical proficiency (MP) level. This was followed by the creation of the items of measurement test according to the proficiency levels. Next, the researchers allocated scoring measures and formed the conversion of learners’ MP stages. Lastly, the researchers validated the superiority of the measurement test through Wright map consuming the multi-dimensional random coefficient multinomial logit model. The construct maps of the MP level consisted of two dimensions, namely mathematical procedures and structure of learning outcome. Findings discovered that there were 20 items in the assessment tool and its quality passed the determined education and psychological assessment criteria. It can be determined that every item is capable to measure the learners’ multi-dimensional mathematical proficiencies.

Using Machine Learning to Score Multidimensional Assessments of Students’ Skill Levels in Mathematics

This research aims to establish a mathematical skill measurement model to examine seventh-grade students’ mathematical skills in two aspects: their understanding of mathematical processes and the concept and structure. The researchers surveyed the mathematical skills of 521 seventh-grade students from the northeastern province of Thailand. Their test results were used to prototype a mathematical skill measurement model using machine learning. It involved a design-based approach that included four stages: a construct map, item design, a Wright Map, and outcome space, the so-called Multidimensional Random Coefficient Multinomial Logit Model, to verify its quality. The initial findings revealed the creation of a construct map consisting of five levels. The researchers determined the cut-off point in the form of the threshold level after considering the Wright Map criteria area for each aspect. Lastly, the measurement model was examined to provide adequate evidence of the internal structure’s validity and reliability. In conclusion, students’ skill levels can be measured accurately using multidimensional assessments, even though the levels of mathematical capabilities of the students varied from low to moderate to high. Therefore, it provides significant evidence of the mathematical skill measurement model to diagnose seventh-grade students’ learning. The significant implications contributed to educational measurement and evaluation are that machine learning algorithms can provide more accurate and consistent scoring of assessments compared to human graders. With accurate assessment using machine learning, teachers can gain deeper insights into individual students’ mathematical skills across multiple dimensions.

Modeling Dimensions Converging at the Upper Anchor in Learning Progressions: An Example of Micro-Evolution

ABSTRACT Multidimensionality is common in psychological and educational measurements. This study focuses on dimensions that converge at the upper anchor (i.e. the highest acquisition status defined in a learning progression) and compares different ways of dealing with them using the multidimensional random coefficients multinomial logit model and scale alignment methods. Assumptions underlying the four approaches studied are a) ignoring the convergence, b) recognizing the convergence of the dimensions, c) treating convergence as a new dimension, and d) separating the within-category multidimensionality and treating convergence as a new dimension. A learning progression about micro-evolution is used as an example, including model fits, step difficulties, and associations between dimensions, Wright maps are drawn, and inferences are made under the four building blocks of measurement development. Finally, the usefulness and weaknesses of the four approaches are discussed.

Use of Digital Learning Platform in Diagnosing Seventh Grade Students’ Mathematical Ability Levels

This paper aims to describe the design and inspection of the quality of a digital learning platform to diagnose mathematical ability levels of seventh-grade students with regard to the topics of Measurement and Geometry. A total of 517 seventh-grade students from 23 schools in four regions, namely north, northeast, central, and south of Thailand were randomly chosen took part as test-takers. The researchers employed a design-based research approach incorporating three stages starting from designing a digital learning platform as an assessment model to diagnose students’ mathematical ability levels, up to employing a Multidimensional Random Coefficient Multinomial Logit Model to inspect the quality of the seventh-grade students’ mathematical abilities assessment model. The research tool consisted of three subjective questions and 15 multiple-choice questions which were used to develop two five-level construct maps ranging from unresponsive to tactical intelligent for the mathematical procedure dimension, and prolonged intellectual structure for the conceptual structural dimension. The findings verified that there were three forms of evidence to support the quality of the mathematical abilities assessment model. The results of the intersection of a mathematical procedure and conceptual structural dimensions’ transition point from levels 1 to 5 as ranging from the lowest to the highest levels at -1.41, -0.69, 0.49, 1.34 and -0.98, 0.14, 0.44, 1.70 respectively. Finally, the empirical findings indicated that the digital learning platform was at a highly appropriate level in terms of its usefulness, suitability, and accuracy except for in terms of feasibility which was at a moderately appropriate level. In conclusion, the digital learning platform can be used to provide substantial information when it comes to diagnosing seventh-grade students’ mathematical ability levels.

Designing Standards-Setting for Levels of Mathematical Proficiency in Measurement and Geometry: Multidimensional Item Response Model

This study intends to design and verify the quality of a model that measures mathematical proficiency and aims to set the standards in measuring levels of proficiency in the subjects of measurement and geometry. Construct modeling was employed to design a mathematical proficiency measurement model which consists of the mathematical process and the dimensions of a conceptual structure. A total of 517 Secondary Year 1 students were selected from the big data to participate as test-takers. Design-based research encompassing four phases was used to verify the quality of the mathematical proficiency measurement model. A Multidimensional Random Coefficient Multinomial Logit model was used to examine the standards-setting of the mathematical proficiency measurement model. The results indicated that the two dimensions of mathematical proficiency can be further divided into five levels, from non-response/irrelevance to strategic/extended thinking and extended abstract structure for mathematical process and conceptual structural dimensions, respectively. The assessment tool covers 18 items with 15 multiple-choice items and three subjective items in measurement and geometry. Moreover, the results also demonstrated that the validity evidence associated with the internal structure of the multidimensional model is fit. Besides, reliability evidence, as well as item fit, is compliance with the quality of the mathematical proficiency measurement model as illustrated in analysis of the standard error of measurement and infit and outfit of the items. Finally, the researchers managed to set standards for the mathematical proficiency measurement model based on the assessment criterion results from the Wright Map. In conclusion, the standards-setting of the mathematical proficiency measurement model provides substantial information, particularly for measuring those students who are above the lowest level of mathematical proficiency because the error for estimating proficiency was low.

Computer Adaptive Test Development To Assess Students’ Psychology

Stress becomes a significantly serious issue among university students and we need efficient tools to understand it more. The aim of present study is to develop a Computerized Adaptive Testing (CAT) to measure the mentioned stress, as pioneer project in Vietnam. In this vein, an item bank of 68 items has been constructed, which is based on Likert Polytomous Scales through five subdomains: behavior, academic performance, family, lecturer and finance. The sampling of the survey is large. It has assessed 2,085 students (704 males and 1,381 females). Multidimensional Random Coefficients Multinomial Logit (MRCML) Model is applied to develop Multidimensional Stress Scales and Computerized Adaptive Testing procedure. The result findings indicate that Multidimensional Random Coefficients Multinomial Logit (MRCML) can be used to develop new scale with psychometric properties. Indicated by various fit criteria MNSQ, standard errors, Z (t-test) implemented in software ConQuest. The subdomain has a good reliability (from .857 to .798). As respect to CATs, a simulated experiment based on the empirical data is applied to evaluate the performance of the proposed computerized adaptive testing. The standard error of the estimated stress proficiencies are reported in this study. The 68 items stress data appropriate fit the Multidimensional model applied.

Developing an assessment framework of multidimensional scientific competencies

The study aimed to develop and validate an assessment framework of multidimensional scientific competencies for seventh-grade students in the northeastern region of Thailand. A total of 289 samples with three different scientific competency levels were randomly selected to participate as testtakers. The design-based research encompassing four phases of the construct modelling approach, namely construct maps, item design, outcome space, and Wright map. Multidimensional Random Coefficient Multinomial Logit model was employed to examine the quality of the created assessment framework of multidimensional scientific competencies. The results showed that scientific competence is comprised of three dimensions, namely explain phenomena scientifically, evaluate and design scientific inquiry, and interpret data and evidence scientifically. Each dimension can be further categorized into four levels. The assessment framework consists of 16 items. The results revealed that there is validity evidence regarding internal structure based on the comparison of the model fit and Wright map. Moreover, results also indicated that the reliability evidence and item fit are compliance with the quality of the assessment framework as revealed in the analysis of standard error of measurement and infit and outfit of the items. It can be concluded that the assessment framework is currently prevalent to assess the scientific competencies of seventh-grade students.

Designing and verifying a tool for diagnosing scientific misconceptions in genetics topic

The main purpose of this study was to design and verify the quality of an assessment aimed at diagnosing scientific misconceptions among students enrolled in 10th-grade biology subjects. A sample consisted of N=200 students from schools under the administration of the Office of Secondary Educational Service 31, Nakhon Ratchasima province. We employed the design-based approach, which consists of four phases, namely, construct map, item design, outcome space, and Wright map. Multidimensional Random Coefficient Multinomial Logit Model (MRCMLM) was used to evaluate the quality of the assessment tool. The assessment tool consists of two dimensions, namely, knowledge and reasoning. The assessment tool is comprised of 40 items, with 20 items tapping each of the dimensions. Findings form item analysis and modeling revealed sufficient evidence on the internal structure and validity of the instrument. It can be concluded that the unidimensional model is more appropriate than the multidimensional model for diagnosing scientific misconceptions in the genetics topic for dichotomously scored with four options in each item. The polytomous scored type should be further investigated to determine the best fit model in the items relating to evidence-based reasoning for supporting students’ answers.

Constructing Progress Maps of Digital Technology for Diagnosing Mathematical Proficiency

This research aims to construct and validate progress maps of digital technology for diagnosing the multidimensional mathematical proficiency (MP) in Number and Algebra for Grade 7 students utilizing the Construct Modeling Approach. Researchers employed four building blocks as follows. Firstly, researchers developed the progress maps as an assessment framework of multidimensional MP. This is followed by creating the test for diagnosing MP. Next, researchers assigned scoring criteria and created the transition points of students’ MP levels. Finally, researchers validated the quality of the progress maps through empirical evidence. A total sample 1,500 Grade 7 students was used to support the validity and reliability evidence of the progress maps through the Wright Map using Multidimensional Random Coefficients Multinomial Logit Model. Results revealed that there were two dimensions of progress maps, namely mathematical procedures (MAP) and structure of learning outcome (SLO), and the researchers investigated three strands of validity evidence, namely test content, response processes, and internal structure. The reliability values in the MAP and SLO were 0.84 and 0.80 respectively. Finally, the Grade 7 students were mainly found to be at level-2 in the MAP dimension (44.95%) and the SLO dimension (61.57%). The experts’ evaluation results showed that the digital technology that was developed at the “most appropriate” quality levels in terms of usefulness, suitability, and accuracy, and at the “very appropriate” for the feasibility aspect, and hence is successfully contributing to the clarification of learning goals, to support for student-centered instruction, and that it is helpful in improving in teacher professional development.

A Systematic Review of Instruments Measuring Family and Social Support of Breastfeeding Mothers

Background: Due to low rate of breastfeeding among working mothers and of interventions to increase duration of breastfeeding, this systematic review conducted to evaluate psychometric properties of instruments measured mother’s perception of Breastfeeding Support. Materials and Methods: The search was carried in English language databases including Medline (via PubMed), Scopus, Cochran library and Web of Science since inception to March 2018 regarding published studies evaluating psychometric properties of Breastfeeding Self-Efficacy. The COSMIN checklist was used to assess quality of related studies. Results Authors of Perceived Breastfeeding Support Assessment Tool’ (PBSAT) suggested that instrument seem to should be two workplace environmental for breast-feeding working mother and the available social environmental for working mothers. Total Cronbach's alpha was 0.85. In exclusive breastfeeding social (EBFSS) instrument, based on exploratory factor analysis, 16 items grouped into three instrumental, emotional and informational factors accounted 66% of total variance. EFA were followed by confirmatory analysis showed Modified model was partially fitted to data. In Workplace Breastfeeding Support Scale (WBSS), EFA identified four dimensions of breastfeeding at workplace. These four labeled technical support, breastfeeding-friendly environment, facility support and peer support. Cronbach’s alpha was 0.77 and split-half reliability was r=0.86. In Employee Perceptions of Breastfeeding Support Questionnaire (EPBS-Q), data scaled by Multidimensional Random Coefficients Multinomial Logit Model. A two-dimensional model (company polices/work culture and manager and her co-workers) were emerged. Cronbach’s alpha was excellent (almost 0.90). Conclusion: Four instruments found to assess breastfeeding was valid and reliable to measure breast feeding in social and workplaces.

Ajuste de un modelo Rasch multidimensional al cuestionario de las cinco facetas de conciencia plena (FFMQ) aplicado a practicantes de yoga

This research aims to gather evidence on the validity of the internal structure of and estimate the reliability of the measurements from the FFMQ (Baer et al., 2006) using the multidimensional random coefficient multinomial logit model. Such instrument consists of 39 Likert-type items answered by selecting an option from a five-point rating scale on five dimensions: Observing, Describing, Acting with Awareness, Non-Judging and Non-Reactivity to Inner Experience. The analysis was conducted using data collected from an accidental non-probabilistic sample of 288 yoga practitioners who reported not being under a psychotherapeutic or psychopharmacological treatment.

Multidimensional Analysis of the Arabic Version of the Actual Scope of Nursing Practice Scale.

Data were analyzed from a national convenience sample of 3,000 bedside nurses in Lebanon to evaluate the psychometric characteristics of the first Arabic version of the Actual Scope of Nursing Practice (A-ASCOP) questionnaire. The method used in this study was application of the partial credit model using the multidimensional random coefficients multinomial logit model. A-ASCOP subscales (r = .81-.94) and levels of item complexity (r = .95-.98) were highly correlated. As a 26-item scale, the A-ASCOP has high internal consistency (Cronbach's α = 0.93). The A-ASCOP subscales have acceptable multidimensional reliability Expected A-Posteriori/Plausible Value ratio (EAP/PV > .80) and are suitable for descriptive surveys. The three A-ASCOP levels of item complexity were not valid for this sample. We report norms for unidimensional Rasch and 0-100 transformed measures by category of participant.

Ajuste de modelos Rasch multidimensionales, de testlets y de diagnóstico cognitivo a las Pruebas ECE 2015

This study aimed at applying the results of three models: a) the Multidimensional Random Coefficients Multinomial Logit Model, b) the Rasch Testlet Model, and c) the Cognitive Diagnosis Model, and comparing those results to the data generated from the math and reading comprehension tests conducted in the 2015 Student Census Evaluation (ECE). The sample consisted of 5,000,000 evaluations of students attending second grade of primary school in educational institutions belonging to the regular basic education program (Minedu, 2009). The results showed that: 1. there is a high correlation between modeled dimensions in the multidimensional model both in the math and reading comprehension tests, 2. it is convenient to apply a one-dimensional model to the data derived from the application of the ECE tests, and 3. there is some consistency between the Rasch Testlet Model and the Cognitive Diagnosis Model.

The Application of Multidimensional Rasch Models in Large Scale Assessment and Validation: An Empirical Example

Introduction. Accountability and diagnostic assessments usually comprise several subtests to provide information on the different components of the abilities tested. For practicality reasons such subtests are usually short. The major drawback of short subtests is their low reliability and imprecision. This study shows how a compensatory multidimensional Rasch model, namely, Multidimensional Random Coefficient Multinomial Logit Model (MRCMLM), can utilize the correlations among subtests to improve estimation and the reliability of measures.Method. The sample was composed of 1021 Iranian high school students. An English language comprehension test composed of four 10-item subtests was adminstretd to the participants. The data were analysed with both MRCMLM and unidimensional Rasch model.Results. Findings showed that subtests that contain only ten items can have the reliability of 45-item tests when students’ abilities are estimated with the MRCMLM. It is also demonstrated that multidimensional analysis gives a better estimate of the true correlations among subtests and can be used as a confirmatory approach to study the componential structure of proficiency constructs.Conclusions. Multidimensional Rasch model can improve the reliability of short subtests. The model can also be used as a confirmatory approach to study the componential structure of proficiency constructs and to show the true association among dimensions of profificency.

Evaluating the metacognitive awareness inventory using empirical factor-structure evidence

Many scholars agree on the general theoretical structure of metacognition, which is what informed the development of the Metacognitive Awareness Inventory (MAI). Although self-report instruments such as the MAI suffer many threats to validity, they continue to be used in research and practice because of their convenience. With the MAI, studies have varied in the way they calculate scores and in their adherence to the intended theory. In this study, we address these shortcomings and propose modifications in calculating MAI scores. Using confirmatory factor analysis (CFA) and multidimensional random coefficients multinomial logit (MRCML) item-response modeling, we examined how well the intended functioning of the MAI matched the data from 622 undergraduate students. The results support scoring the MAI as two dimensions, knowledge and regulation of cognition, but indicate that the 52-item instrument has poor fit. Using iterative CFA and MRCML models, we tested subsets of items that represent the theory and had good fit. We followed up with tests of between-group and time invariance. The results support the use of a 19-item subset for between-group comparisons, with provisional evidence for its use in longitudinal studies.

Validating the European Health Literacy Survey Questionnaire in people with type 2 diabetes: Latent trait analyses applying multidimensional Rasch modelling and confirmatory factor analysis.

To validate the European Health Literacy Survey Questionnaire (HLS-EU-Q47) in people with type 2 diabetes mellitus. The HLS-EU-Q47 latent variable is outlined in a framework with four cognitive domains integrated in three health domains, implying 12 theoretically defined subscales. Valid and reliable health literacy measurers are crucial to effectively adapt health communication and education to individuals and groups of patients. Cross-sectional study applying confirmatory latent trait analyses. Using a paper-and-pencil self-administered approach, 388 adults responded in March 2015. The data were analysed using the Rasch methodology and confirmatory factor analysis. Response violation (response dependency) and trait violation (multidimensionality) of local independence were identified. Fitting the "multidimensional random coefficients multinomial logit" model, 1-, 3- and 12-dimensional Rasch models were applied and compared. Poor model fit and differential item functioning were present in some items, and several subscales suffered from poor targeting and low reliability. Despite multidimensional data, we did not observe any unordered response categories. Interpreting the domains as distinct but related latent dimensions, the data fit a 12-dimensional Rasch model and a 12-factor confirmatory factor model best. Therefore, the analyses did not support the estimation of one overall "health literacy score." To support the plausibility of claims based on the HLS-EU score(s), we suggest: removing the health care aspect to reduce the magnitude of multidimensionality; rejecting redundant items to avoid response dependency; adding "harder" items and applying a six-point rating scale to improve subscale targeting and reliability; and revising items to improve model fit and avoid bias owing to person factors.

Components of quality of life in a sample of patients with lupus: a confirmatory factor analysis and Rasch modeling of the LupusQoL.

The objective of this study was to test different exploratory solutions to the LupusQoL scale in a sample of Spanish patients with SLE using confirmatory factor analysis (CFA) and Rasch modeling, as well as to estimate the convergent validity. The χ 2 test, RMSEA, CFI, and TLI were used to test the fit of the different exploratory structures with CFA. To estimate the parameters in the dimensions found, a rating scale Rasch multidimensional random coefficient multinomial logit model was used. The reliability of the scores was obtained with coefficient alpha and coefficient omega. The convergent validity was calculated using Spearman's rho. Four hundred and fifty patients participated but complete data were available for 223 subjects. The original version (UK) and the French version obtained the best fit, showing that the proposed original structure was the best solution for the structure of the LupusQoL scale in the Spanish sample. The multidimensional solution of eight dimensions was adequate, but item 8 in physical health, item 16 in intimate relations, and items 29 and 30 obtained mean squares >1.6. Internal consistency and coefficient omega of the scores in the eight domains were higher. The Spanish version of LupusQoL correlated strongly with the corresponding SLAQ, EQ5D analogic scale, and EQ5D domain. This analysis confirmed the structure of eight dimensions of the original version in patients with SLE.

Multidimensional Computerized Adaptive Testing for Indonesia Junior High School Biology

Background:This paper describes a curriculum-based multidimensional computerized adaptive test that was developed for Indonesia junior high school Biology.Materials and methods:In adherence to the Indonesian curriculum of different Biology dimensions, 300 items was constructed, and then tested to 2238 students. A multidimensional random coefficients multinomial logit model was used to develop multidimensional scales.Results:A simulation study based on the item and ability parameters estimated from real data is conducted to evaluate the performance of the proposed Biology-MCAT system. The results show that the bias and standard error of Biology-MCAT system are acceptable.Conclusions:It suggests that, this online assessment system can be a model for Indonesian National Examination in future.

Depression Symptoms of Mothers and Fathers of Persons with Schizophrenia

The purpose of the present study was to investigate if depression symptomatology of patients' parents is predicted by the symptoms of schizophrenia. 101 mothers and 101 fathers of the same patients suffering from schizophrenia were included into this study. Parents filled in the "Beck Depression Inventory". Patients were assessed by means of the "Positive and Negative Syndrome Scale". For statistical analyses a Multidimensional Random Coefficients Multinomial Logit Model was applied. We found a significant positive association between negative symptoms and depression severity of fathers and mothers. Further, a significant positive association between positive symptoms and depression severity of fathers, but not of mothers was found. Our results show that depression of mothers and of fathers is associated with symptoms of schizophrenia even when controlling for potential predictors.