Item Response Models Research Articles

Analytical Approximation of the Jackknife Linking Error in Item Response Models Utilizing a Taylor Expansion of the Log-Likelihood Function

Linking errors in item response models quantify the dependence on the chosen items in means, standard deviations, or other distribution parameters. The jackknife approach is frequently employed in the computation of the linking error. However, this jackknife linking error could be computationally tedious if many items were involved. In this article, we provide an analytical approximation of the jackknife linking error. The newly proposed approach turns out to be computationally much less demanding. Moreover, the new linking error approach performed satisfactorily for datasets with at least 20 items.

Rasch model for the need assessment instrument of academic guidance and counseling program in junior high school

Developing an instrument for the need for academic guidance and counseling topics is a strategic step to support the needs assessment carried out by guidance and counseling teachers and counselors in schools. The development of this instrument had been carried out by Arfalah using exploratory factor analysis. This study aims to analyze the instrument using the Rasch model. The research procedure began with collecting data from two integrated online needs assessments of academic guidance and counseling instruments. The content validity test involved six experts, and the field test involved 111 students from three junior high schools in Yogyakarta. Data were analyzed using Winsteps version 3.73 according to Fisher’s five criteria for a good instrument. The content validity test showed Aiken’s coefficient of 0.71, considered in the high category. Meanwhile, the results of the Rasch test showed the item response model. The analysis also indicated high reliability on both instruments above 0.90 and Cronbach’s alpha value of 0.98. Five items were decided to be maintained based on the misfit test explored. The accuracy and focus of the instruments in revealing the need for academic guidance and counseling topics for high school students indicated a number above 40% and a fairly good level of instrument contamination, which was below 10%.

PENGEMBANGAN INSTRUMEN TES PENGUASAAN KONSEP REPRESENTASI KIMIA PADA LIMA INDIKATOR ASAM BASA DARI ALAM : ANALISIS DENGAN RASCH MODEL

This study developed a test instrument for mastering the concept of chemical representation on five natural acid-base indicators. This test aims to determine the mastery of macroscopic, submicroscopic and symbolic chemical concepts. Product trial data processed using the Rasch model assisted by Winstep software. The result showed that the instrument was valid as evidenced by a unidimensional test, person reliability was in good category, item reliability was in sufficient category, Cronbach alpha was in very good category. questions, the value of separation person is divided into three groups, the developed test instrument meets the criteria for the level of difficulty index according to the Rasch Model item response. All items are stated to have good accuracy and can measure students' abilities well.

NEPSscaling: plausible value estimation for competence tests administered in the German National Educational Panel Study

Educational large-scale assessments (LSAs) often provide plausible values for the administered competence tests to facilitate the estimation of population effects. This requires the specification of a background model that is appropriate for the specific research question. Because the German National Educational Panel Study (NEPS) is an ongoing longitudinal LSA, the range of potential research questions and, thus, the number of potential background variables for the plausible value estimation grow with each new assessment wave. To facilitate the estimation of plausible values for data users of the NEPS, the R package NEPSscaling allows their estimation following the scaling standards in the NEPS without requiring in-depth psychometric expertise in item response theory. The package requires the user to prepare the data for the background model only. Then, the appropriate item response model including the linking approach adopted for the NEPS is selected automatically, while a nested multiple imputation scheme based on the chained equation approach handles missing values in the background data. For novice users, a graphical interface is provided that only requires minimal knowledge of the R language. Thus, NEPSscaling can be used to estimate cross-sectional and longitudinally linked plausible values for all major competence assessments in the NEPS.

Continuous norming in learning progress monitoring-An example for a test in spelling from grade 2-4.

One of the main goals of the teacher and the school system as a whole is to close learning gaps and support children with difficulties in learning. The identification of those children as well as the monitoring of their progress in learning is crucial for this task. The derivation of comparative standards that can be applied well in practice is a relevant quality criterion in this context. Continuous normalization is particularly useful for progress monitoring tests that can be conducted at different points in time. Areas that were not available in the normalization sample are extrapolated, closing gaps in applicability due to discontinuity. In Germany, teachers participated in a state-funded research project to formatively measure their children's spelling performance in primary school. Data (N = 3000) from grade two to four were scaled, linked and translated into comparative values that can be used in classrooms independently from specific times. The tests meet the requirements of item response models and can be transferred well to continuous norms. However, we recommend using the 10th or 20th percentile as cut-off points for educational measures, as the 5th percentile is not discriminating enough.

Effects of Affix Type and Base Word Transparency on Students' Performance on Different Morphological Awareness Measures

We examined whether affix type and base word transparency explained variation in third- through sixth-grade students' performance on a number of morphological awareness tasks. Third- through sixth-grade students (n > 500 at each grade) completed morphological awareness tasks from the Morphological Awareness Test for Reading and Spelling, which represent the ways individuals may use their morphological awareness to support reading and spelling. Explanatory item response models were used to understand the role of affix type and base word transparency on students' performance on six morphological awareness tasks. For all grades, 73%-83% of variance in students' performance was due to differences across individual items. Furthermore, when task effects, affix type, and base word transparency were included simultaneously in the model, affix type was not a significant predictor; there was a significant effect of base word transparency and task. Specifically, the probability of a correct response was greater on task items in which inflected or derived words were transparent with their base word (e.g., friend > friendly) compared to items in which there was a shift in both the phonological and orthographic aspects of the base word (e.g., attend > attention). These findings emphasize the importance of considering base word transparency when assessing students' morphological awareness skills with less emphasis on affix type, at least for third- through sixth-grade students. Our results also point to the importance of administering a variety of morphological awareness tasks to fully capture an individual's morphological awareness skills. Collectively, researchers and practitioners should ensure assessment items on multiple measures of morphological awareness vary in their base word transparency to potentially capture a range of student performances.

Validation of response processes in medical assessment using an explanatory item response model

Response process validation is a crucial source of test validity. The expected cognitive load scale was created based on the reflection of the mental effort by which borderline students solve an item defined by experts. The stem length affects the students' extraneous cognitive load. The purposes of this study were to develop an exam for medical students and corroborate the response process validity by analyzing the correlation between the expected cognitive load, stem length, and the difficulty. This was a correlational study. Five medical teachers as the experts and 183 third-year medical students were enrolled from the Faculty of Medicine, Prince of Songkla University, Thailand. The instruments used were a medical physiology exam and a three-level expected cognitive load evaluation form judged by medical teachers. Data were analyzed using an explanatory item response model. The test consists of 20 items and 21 possible scores. The median score was 8, with a quartile deviation of 1.5. Nine items had long stems (more than two lines). Sixteen items were judged as high (level 2 or 3) expected cognitive load. When adding the expected cognitive load in a Rasch model, the expected cognitive load significantly correlated with item difficulty. In the Rasch model that included both the expected cognitive load and stem length, a long stem had a greater effect on item difficulty than low expected cognitive load. However, the Rasch model showed the best fit. The long stem had a stronger correlation with test difficulty than expected cognitive load, which indirectly implied response process validity. We suggest incorporating stem length and expected cognitive load to create an appropriate distribution of the difficulty of the entire test.

Application of Asymmetric Item Response Models to a Vertical Scaling Context

Application of Asymmetric Item Response Models to a Vertical Scaling Context

Activity and participation in haemophiliacs: Item response modelling based on international classification of functioning, disability and health.

There is scant research investigating the user-friendly functional assessment tool conceptualized by the International Classification of Functioning, Disability and Health (ICF) among persons with haemophilia (PWH). This study aims to accomplish two goals: (1) quantifying comprehensive functioning measures of haemophilia through Item Response Theory (IRT); (2) discussing patient-centred care based on the Wright map of personal ability and item difficulty. A cross-sectional study was carried out in 70 PWH (mean age, 33.09±11.04) via convenience sampling. All patients completed the 45 ICF categories of haemophilic-specific activity and participation. Psychometric properties of the categories were examined using Mokken scale analysis and parametric item response modelling. We extracted a unidimensional scale with 31 categories, and constructed a Rasch model with good fitness. The Cronbach's α of the scale was .9713, with the Guttman's λ2 =.9730, Molenaar Sijtsma ρ=.9802, and latent class reliability coefficient=.9769, indicating great internal reliability. The estimated individual social competence by the Rasch model was highly related to the index score of the three-level EuroQol five-dimensional questionnaire (EQ-5D-3L) (p<.001, r=.62), and had a moderate correlation (p<.001, r=.54) with the score of Haemophilia Activities List (HAL). The ICF scale of haemophilicactivity and participation with 31 categories (HAPPY-ICF) has good construct validity and internal consistency. The person-item threshold distribution map might be helpful in research and clinical practices for patient-oriented care.

A stochastic approximation expectation maximization algorithm for estimating Ramsay-curve three-parameter normal ogive model with non-normal latent trait distributions.

In the estimation of item response models, the normality of latent traits is frequently assumed. However, this assumption may be untenable in real testing. In contrast to the conventional three-parameter normal ogive (3PNO) model, a 3PNO model incorporating Ramsay-curve item response theory (RC-IRT), denoted as the RC-3PNO model, allows for flexible latent trait distributions. We propose a stochastic approximation expectation maximization (SAEM) algorithm to estimate the RC-3PNO model with non-normal latent trait distributions. The simulation studies of this work reveal that the SAEM algorithm produces more accurate item parameters for the RC-3PNO model than those of the 3PNO model, especially when the latent density is not normal, such as in the cases of a skewed or bimodal distribution. Three model selection criteria are used to select the optimal number of knots and the degree of the B-spline functions in the RC-3PNO model. A real data set from the PISA 2018 test is used to demonstrate the application of the proposed algorithm.

Four-Parameter Guessing Model and Related Item Response Models

Guessing effects frequently occur in testing data in educational or psychological applications. Different item response models have been proposed to handle guessing effects in dichotomous test items. However, it has been pointed out in the literature that the often employed three-parameter logistic model poses implausible assumptions regarding the guessing process. The four-parameter guessing model has been proposed as an alternative to circumvent these conceptual issues. In this article, the four-parameter guessing model is compared with alternative item response models for handling guessing effects through a simulation study and an empirical example. It turns out that model selection for item response models should be rather based on the AIC than the BIC. However, the RMSD item fit statistic used with typical cutoff values was found to be ineffective in detecting misspecified item response models. Furthermore, sufficiently large sample sizes are required for sufficiently precise item parameter estimation. Moreover, it is argued that the criterion of the statistical model fit should not be the sole criterion of model choice. The item response model used in operational practice should be valid with respect to the meaning of the ability variable and the underlying model assumptions. In this sense, the four-parameter guessing model could be the model of choice in educational large-scale assessment studies.

A note on the use of rank-ordered logit models for ordered response categories.

Models for rankings have been shown to produce more efficient estimators than comparable models for first/top choices. The discussions and applications of these models typically only consider unordered alternatives. But these models can be usefully adapted to the case where a respondent ranks a set of ordered alternatives that are ordered response categories. This paper proposes eliciting a rank order that is consistent with the ordering of the response categories, and then modelling the observed rankings using a variant of the rank ordered logit model where the distribution of rankings has been truncated to the set of admissible rankings. This results in lower standard errors in comparison to when only a single top category is selected by the respondents. And the restrictions on the set of admissible rankings reduces the number of decisions needed to be made by respondents in comparison to ranking a set of unordered alternatives. Simulation studies and application examples featuring models based on a stereotype regression model and a rating scale item response model are provided to demonstrate the utility of this approach.

The Development of a Standardized Effect Size for the SIBTEST Procedure

In this study a standardized effect size was created for use with the SIBTEST procedure. Using this standardized effect size, a single set of heuristics was developed that are appropriate for data fitting different item response models (e.g., 2-parameter logistic, 3-parameter logistic). The standardized effect size rescales the raw beta-uni value using a pooled variation that incorporates the beta-uni inclusion factor. Although the heuristics for the standardized and unstandardized effect sizes provide similar true-positive and false-positive rates in most conditions, the standardized effect size provides higher true-positive rates for conditions where item response variability is smaller in proportion to raw score differences. Inflated false-positive rates were solely impacted by smaller sample sizes, whereas larger sample sizes improved true-positive rates. An empirical application is provided to demonstrate how the standardized effect size provides for a more consistent comparison across items with varying response distributions. This study lays the foundation for the utilization of a standardized effect size for both dichotomous and polytomous item response models using the suite of SIBTEST procedures.

Application of a joint latent space item response model to clustering stressful life events and the Beck Depression Inventory-II: results from Korean epidemiological survey data.

According to previous findings, stressful life events (SLEs) and their subtypes are associated with depressive symptoms. However, few studies have explored potential models for these events and incidental symptoms of depression. Participants (3,966 men; 5,709 women) were recruited from the Cardiovascular and Metabolic Diseases Etiology Research Center cohort. SLEs were measured using a 47-item Life Experiences Survey (LES) with a standardized protocol. Depressive symptoms were assessed using the Beck Depression Inventory-II (BDI-II). Joint latent space item response models were applied by gender and age group (<50 vs. ≥50 years old). Among the LES items, death or illness of close relatives, legal problems, sexual difficulties, family relationships, and social relationships shared latent positions with major depressive symptoms regardless of gender or age. We also observed a gender-specific domain: occupational and family-related items. By projecting LES and BDI-II data onto the same interaction map for each subgroup, we could specify the associations between specific LES items and depressive symptoms.

Application of Exploratory Factor Analysis and Item Response Theory to Validate NHANES ADL Scale in Patients Reporting Rheumatoid Arthritis.

Background: Several instruments are used for measuring functional limitations among rheumatoid arthritis (RA) patients. However, these instruments are incongruously assessed for their psychometric properties. The National Health and Nutritional Examination Survey (NHANES) uses a generic questionnaire to assess the activities of daily living (ADL) to measure functional limitations among its participants. The psychometric properties of the NHANES-ADL scale were evaluated using a patient examination and survey data. Methods: NHANES-ADL scale was assessed for its internal consistency and factor structure. Scale reliability was assessed with Cronbach's alpha reliability coefficient. Principal component analysis with Promax rotation was used to obtain factor structure. Confirmatory factory analysis was used to calculate fit indices. The graded item response theory model was used to estimate item discrimination, difficulty, and test information. Results: Our sample included 1132 individuals with RA. Exploratory factor analyses of 19-item NHANES ADL scale produced one factor solution and accounted for 35% of variance. The Cronbach alpha of this scale was 0.92. The results of graded item response model indicated items performing well discriminating high and low level of functional ability. A higher slope (α) reflected stronger ability of items to discriminate across the continuum. Conclusions: The NHANES ADL scale showed good reliability, single dimensionality, and validity in RA patients. Studies should explore its test-retest reliability and its ability to reliably measure functional change over time in the future.

Evidence-Based Practice Knowledge Retention Among Nurse Practitioner Students.

The mastery of clinical scholarship and analytic methods for evidence-based practice (EBP) is a published essential for nurse practitioner (NP) education; however, retention of EBP-related knowledge during NP education remains unknown. We assessed EBP knowledge during a first-semester research course and again during the last semester using an item response model. We found that changes depended on the exam item, with performance dropping on a third of the items. Our findings suggest a need to integrate EBP concepts across curricula (e.g., feedback loops in courses other than the research course) to retain these essential skills.

A Factor Mixture Model for Item Responses and Certainty of Response Indices to Identify Student Knowledge Profiles

AbstractThe certainty of response index (CRI) measures respondents' confidence level when answering an item. In conjunction with the answers to the items, previous studies have used descriptive statistics and arbitrary thresholds to identify student knowledge profiles with the CRIs. Whereas this approach overlooked the measurement error of the observed item responses and indices, we address this by proposing a factor mixture model that integrates a latent class model to detect student subgroups and a measurement model to control for student ability and confidence level. Applying the model to 773 seventh graders' responses to an algebra test, where some items were related to new material that had not been taught in class, we found two subgroups: (1) students who had high confidence in answering items involving the new material; and (2) students who had low confidence in answering items involving the new material but higher general self‐confidence than the first group. We regressed the posterior probability of the group membership on gender, prior achievement, and preview behavior and found preview behavior a significant factor associated with the membership. Finally, we discussed the implications of the current study for teaching practices and future research.

Use of Item Response Models in Assessing the Health Literacy Facet Understanding Health Information for Early Childhood Allergy Prevention and Prevention of COVID-19 Infections by Pregnant Women and Mothers of Infants

Abstract. Appropriate parental health literacy (HL) is essential to preventively maintain and promote child health. Understanding health information is assumed to be fundamental in HL models. We developed N = 67 items (multiple-choice format) based on information materials on early childhood allergy prevention (ECAP) and prevention of COVID-19 infections to assess the parental HL facet Understand. N = 343 pregnant women and mothers of infants completed the items in an online assessment. Using exploratory factor analysis for ordinal data (RML estimation) and item response models (1-pl and 2-pl model), we proved the psychometric homogeneity of the item pool. 57 items assess the latent dimension Understand according to the assumptions of the 1-pl model (weighted MNSQ &lt; 1.2; separation reliability = .855). Person parameters of the latent trait Understand correlate specifically with subjective socioeconomic status ( r = .27), school graduation ( r = .46), allergy status ( r = .11), and already infected with COVID-19 ( r = .12). The calibrated item pool provides a psychometrically sound, construct-valid assessment of the HL facet Understand Health Information in the areas of ECAP and prevention of COVID-19 infections.

Unobserved Potential: Latent Variables and Music Education Research

AbstractThis article describes several latent variable approaches that can support rigorous quantitative inquiry in music education. I provide a definition of latent variables, list several advantages associated with their use for the measurement of constructs, and review three types of latent variables featuring utility for music education scholars: factor models, item response models, and latent class (mixture) models. Each type of model is illustrated with exemplar studies from music education literature. In addition, I report the results of a systematic analysis of latent variables’ use in published music education research. Over the past decade, latent variables were used in only 14.29% of the quantitative research reports in two prominent journals: the Bulletin of the Council for Research in Music Education and the Journal of Research in Music Education. The majority of these latent variable applications (80.56%) were factor models; however, there are myriad additional possibilities. While not appropriate for every study, latent variable models have as-yet unrealized potential to improve the richness and rigor in nearly all areas of music education research.

A Bibliometric Analysis: A Tutorial for the Bibliometrix Package in R Using IRT Literature

The bibliometrix package in R programming language, which is frequently used in bibliometric analysis, was introduced in this research. The article aimed to illustrate the various analyses applied in a bibliometric study. For this purpose, articles containing the "item response theory" (IRT) or "item response modeling" or "item response model" terms in the abstract were searched in the Thomson Reuters Clarivate Analytics Web of Science (WoS at http://www.webofknowledge.com), and bibliometric data was downloaded. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) steps were followed in the study. Data from 3388 IRT-related articles on education and psychology, searched between 2001 and 2021, were used in the study. Data were analyzed with the bibliometrix package. Some of the stages in data analysis were shared with screenshots. As a result of data analysis through the real data set, the author’s keywords related to IRT were item response model, differential item functioning, psychometrics, assessment, measurement, reliability, validity, Rasch model, and measurement invariance. The countries with the highest number of citations in IRT studies were the USA, Canada, Netherlands, United Kingdom, and China, respectively. Turkey ranked 12th in IRT studies with 434 citations. It was thought that bibliometric analysis of articles related to IRT would shed light on researchers in the field of psychometrics.