Polytomous Item Response Theory Research Articles

Exploring the Effects of Collapsing Rating Scale Categories in Polytomous Item Response Theory Analyses: An Illustration and Simulation Study

ABSTRACT Researchers who work with ordinal rating scales sometimes encounter situations where the scale categories do not function in the intended or expected way. For example, participants’ use of scale categories may result in an empirical difficulty ordering for the categories that does not match what was intended. Likewise, the level of distinction between categories may be imprecise, where there are only minimal differences between categories or categories that reflect too wide of a range of the latent variable to be useful. In these cases, researchers sometimes consider combining categories to minimize the impact of these characteristics on the interpretability of scores. Although this practice is relatively common, the psychometric literature provides limited guidance related to specific strategies for and the psychometric consequences of combining categories. We used empirical data from a self-efficacy scale to illustrate rating scale malfunctioning in a real-world context and demonstrate the impact of various category-collapsing schemes on indicators of rating scale functioning. Then, we used a simulation study to consider how these category-collapsing schemes may impact rating scale functioning under different conditions. Overall, our analyses illustrate different approaches to category collapsing and their impact on psychometric results. Our results suggested that aligning the collapsing scheme with the location of the rating scale malfunctioning was generally effective in improving scale performance. We discuss implications for research and practice and identify several directions for further investigation.

Unsupervised item response theory models for assessing sample heterogeneity in patient-reported outcomes measures

PurposeUnsupervised item-response theory (IRT) models such as polytomous IRT based on recursive partitioning (IRTrees) and mixture IRT (MixIRT) models can be used to assess differential item functioning (DIF) in patient-reported outcome measures (PROMs) when the covariates associated with DIF are unknown a priori. This study examines the consistency of results for IRTrees and MixIRT models.MethodsData were from 4478 individuals in the Alberta Provincial Project on Outcome Assessment in Coronary Heart Disease registry who received cardiac angiography in Alberta, Canada, and completed the Hospital Anxiety and Depression Scale (HADS) depression subscale items. The partial credit model (PCM) based on recursive partitioning (PCTree) and mixture PCM (MixPCM) were used to identify covariates associated with differential response patterns to HADS depression subscale items. Model covariates included demographic and clinical characteristics.ResultsThe median (interquartile range) age was 64.5(15.7) years, and 3522(78.5%) patients were male. The PCTree identified 4 terminal nodes (subgroups) defined by smoking status, age, and body mass index. A 3-class PCM fits the data well. The MixPCM latent classes were defined by age, disease indication, smoking status, comorbid diabetes, congestive heart failure, and chronic obstructive pulmonary disease.ConclusionPCTree and MixPCM were not consistent in detecting covariates associated with differential interpretations of PROM items. Future research will use computer simulations to assess these models’ Type I error and statistical power for identifying covariates associated with DIF.

The effect of real word stimuli versus non-word stimuli on oral diadochokinetic rates across the life span: An item discrimination analysis.

Diadochokinetic (DDK) rate tasks are extensively used in the evaluation of speech disorders; however, it is unclear how the different types of speech stimuli affect DDK rate performance. To investigate the effect of age, gender and type of stimuli (non-words versus real words) on the DDK rates in individuals across the lifespan and to provide normative data for Greek. Also to examine the discrimination ability of the speech DDK stimuli administered (non-words and real words) based on a dual DDK assessment protocol using a polytomous item response theory (IRT) model. The participants were 1747 monolingual Greek speakers (376 children, aged 4-17 years; and 1371 adults, aged 18-90+ years). All participants had normal hearing acuity which allowed them to understand and follow instructions. Participants with a medical condition or a language disorder which would affect DDK rate performance were excluded from the study. The time-by-count method was used, and all participants had to repeat as accurately and fast as possible: (1) four disyllabic non-words (/'gaba/, /'taka/, /'kata/, /'baga/), (2) four disyllabic real words (/'kapa/, /'tapa/, /ka'la/, /'paka/) and (3) two trisyllabic non-words (/'pataka/, /'badaga/). All responses were recorded and the speech samples that did not include at least 5 s of correct repetitions were excluded from the analysis. Age affected significantly DDK rates with performance increasing gradually until approximately the age of 40 and then gradually decreasing. Gender had no effect. Overall, there was a significant advantage of disyllabic real word stimuli over disyllabic non-word stimuli and of trisyllabic non-word stimuli over disyllabic non-word stimuli on DDK rates performance. IRT analysis suggested that the data fit the polytomous model reasonably well and all DDK stimuli (real words and non-words) showed a strong relationship (loadings > 0.50) with the latent trait. The current study complements prior research which supports that age and type of stimuli significantly affect DDK rates performance. It is the first study, that testifies to the benefit of real-word stimuli over non-word stimuli on DDK rates across the lifespan in a large representative sample. The implementation of IRT analysis provides empirical evidence about the discrimination ability of the DDK stimuli administered and confirms the reliability of this dual DDK assessment protocol. These findings are valuable for clinicians who work with motor speech disorders. What is already known on this subject Age, type of stimuli (real words versus non-words) and language significantly affect DDK rates performance. Current research strongly suggests the administration of language-specific norms since language-dependent features seem to have a noteworthy effect on the DDK rates, but scarce evidence exists about the discriminatory ability of the DDK speech stimuli commonly administered. What this study adds to the existing knowledge Conflicting findings have been reported about the effect of different types of DDK speech stimuli (real words and non-words) but no study to date has evaluated their discriminatory abilities. The current study is the first to implement a polytomous IRT model to examine this issue. This is also the first study to attempt an investigation of the effect of types of stimuli (real words versus non-words) on a large representative sample across the lifespan (4-90+ years) and to provide normative data for Greek. What are the practical and clinical implications of this work? The present study offers concrete evidence about the advantage of real-word stimuli over non-word stimuli in Greek, as well as normative data for the Greek-speaking populations. Moreover, the IRT analysis testifies to the discriminatory ability of real-word and non-word stimuli affirming the reliability of the present dual DDK assessment protocol as a psychometrically sound measure of DDK ability. The above has significant value for clinicians who work with individuals with motor speech disorders as the protocol can help them with the diagnosis and differential diagnosis of motor speech disorders.

Psychometric properties of the Psy-flex scale: A validation study in a community sample in Korea

This study examined the psychometric properties of the recently developed six-item Psy-Flex among community samples in Korea. Using a cross-sectional design, data were collected from 1059 participants. Three bilingual experts translated the scale to ensure content validity. Factor analysis was employed to confirm the factor structure, and a polytomous item response theory model was used to examine the individual items and entire scale. The theory-based single-factor structure was confirmed using a Korean community sample, and measurement invariances were found across sex and age groups. In addition, the scale showed a moderate relationship with established measures of interest. Furthermore, item and categorical functioning were investigated using the polytomous Item Response Theory (IRT) model (i.e., the Generalized Partial Credit Model [GPCM]), identifying well-functioning (e.g., items 4, 5, and 6) and suboptimal (e.g., item 2) items. Additionally, the results suggest that the participants might not be able to meaningfully differentiate the original five-response categories. To the best of our knowledge, this is the first attempt to validate the Psy-Flex in Korea. We believe that the findings are of considerable value in facilitating our understanding of the scale and, more broadly, the construct of psychological flexibility.

Developing New Scales of Personal Food Security

BackgroundFood insecurity is standardly measured at the household level or for groups of household members. However, food hardships may differ for individuals within households. Summary measures of people’s individual experiences of food insecurity are needed. ObjectiveThis study aims to develop and analyze psychometrically sound multi-item scales of people’s individual experiences of food insecurity. It further aims to examine whether and how the distributions of personal food insecurity differ across age groups, from household food insecurity, and with people’s observed characteristics. DesignThe study analyzes questionnaire data on personal food security outcomes, household food security outcomes, and other characteristics. Participants and settingThe 29,040 participants in the study responded to the 2005-2006 through 2009-2010 cycles of the National Health and Nutrition Examination Survey in the United States. Main outcomesThe main outcomes are five-item scales of personal food insecurity for children younger than age 12 years, young adolescents aged 12 to 15 years, and people aged 16 years and older. Statistical analysis performedThe study develops the personal food insecurity scales through factor analyses and polytomous Item Response Theory models and analyzes characteristics that are related to the scale outcomes through multivariate regressions. ResultsThe article develops personal food insecurity scales that are related to but distinct from the standard household scales, with different scales being needed to capture the experiences of its three age groups. Children younger than age 12 years have much lower risks of personal food insecurity than other age groups, whereas young adolescents have higher risks than other groups. Three percent of children had an indication of personal food insecurity, whereas 10% of adolescents and 7% of adults had indications. Among adults, women and people between ages 31 and 65 years have higher risks of personal food insecurity than men and people of other ages, but they have similar risks for household food insecurity. ConclusionsPersonal food insecurity is a distinct component of well-being that can be summarized through scale measures. Evidence that characteristics, such as sex and age, are related to personal food insecurity but not household food insecurity indicates that food experiences can differ within households and that some people may be systematically disadvantaged.

Establishing the measurement precision of the patient health questionnaire in an adolescent sample

BackgroundTechnically sound measures are necessary for accurately identifying youth at risk for depression, but many studies rely on classical test theory metrics or adult samples to evaluate measures. This study examined the use of the PHQ-8, a common and freely available pediatric depression screener, in an adolescent sample using item response theory (IRT). MethodsSecondary analyses were conducted on data from a study conducted in Midwestern middle schools in which 1224 youth completed the PHQ-8 as part of a battery of surveys. Polytomous IRT analyses (a Graded Response Model) were used to evaluate the PHQ-8. Items were examined for their ability to distinguish between respondents of different latent depression severity and for differential item functioning (DIF) across demographic categories. ResultsAll PHQ-8 items had adequate discriminative abilities. Items measuring anhedonia and psychomotor disturbances performed relatively poorly, and items measuring somatic symptoms (appetite and sleep) were most informative when respondents endorsed extreme response options (“not at all” or “nearly every day”). No DIF was found across grade level or race, but several items were flagged for DIF by gender and student income level. LimitationsThese results might not be generalizable to a broader youth population due to administration setting and the unique demographic characteristics of this sample (76.0 % African American). ConclusionsTools such as the PHQ-8 are appropriate to quickly screen for depression in adolescents, but further scrutiny of adolescent response patterns is warranted. Future research should examine items measuring anhedonia and psychomotor and somatic disturbances in adolescents.

The Effect of Option Differences on Psychometric Properties of Items in Likert-Type Scales

Likert-type scales are often used in education and psychology. In Likert-type scales, response options/categories, like items, are expected not to direct individuals’ responses. Although the researchers themselves make decision on how to arrange categories during scale development, it is possible that different categories reveal different response behaviors. In the literature, it has been observed that differentiations in the number of categories of forms are studied more, yet there are a limited number of studies investigating the middle category in the forms with different labels. Furthermore, it has also been observed that there are limited number of empirical studies conducted based on polytomous Item Response Theory. This study, which was conducted to close this gap in the literature, was carried out with 377 students. The options of the attitude scale were denominated with different labels, and thus four different forms were generated. Only the middle category names were changed in the first three forms, and in the fourth form, the categories were graded. The data obtained from the forms were analyzed using the Graded Response Model and the Generalized Partial Credit Model depending on Item Response Theory. After the examination of reliability of the forms, the parameters in these forms, and the relationships between the parameters according to both models, inferences were made as to how the differences of the middle category in the forms had an effect on the perceptions of individuals.

Wald χ2 Test for Differential Item Functioning Detection with Polytomous Items in Multilevel Data

Identifying items with differential item functioning (DIF) in an assessment is a crucial step for achieving equitable measurement. One critical issue that has not been fully addressed with existing studies is how DIF items can be detected when data are multilevel. In the present study, we introduced a Lord’s Wald [Formula: see text] test-based procedure for detecting both uniform and non-uniform DIF with polytomous items in the presence of the ubiquitous multilevel data structure. The proposed approach is a multilevel extension of a two-stage procedure, which identifies anchor items in its first stage and formally evaluates candidate items in the second stage. We applied the Metropolis–Hastings Robbins–Monro (MH-RM) algorithm to estimate multilevel polytomous item response theory (IRT) models and to obtain accurate covariance matrices. To evaluate the performance of the proposed approach, we conducted a preliminary simulation study that considered various conditions to mimic real-world scenarios. The simulation results indicated that the proposed approach has great power for identifying DIF items and well controls the Type I error rate. Limitations and future research directions were also discussed.

Development of a character qualities test for medical students in Korea using polytomous item response theory and factor analysis: a preliminary scale development study

This study aimed to develop a test scale to measure the character qualities of medical students as a follow-up study on the 8 core character qualities revealed in a previous report. In total, 160 preliminary items were developed to measure 8 core character qualities. Twenty questions were assigned to each quality, and a questionnaire survey was conducted among 856 students in 5 medical schools in Korea. Using the partial credit model, polytomous item response theory analysis was carried out to analyze the goodness-of-fit, followed by exploratory factor analysis. Finally, confirmatory factor and reliability analyses were conducted with the final selected items. The preliminary items for the 8 core character qualities were administered to the participants. Data from 767 students were included in the final analysis. Of the 160 preliminary items, 25 were removed by classical test theory analysis and 17 more by polytomous item response theory assessment. A total of 118 items and sub-factors were selected for exploratory factor analysis. Finally, 79 items were selected, and the validity and reliability were confirmed through confirmatory factor analysis and intra-item relevance analysis. The character qualities test scale developed through this study can be used to measure the character qualities corresponding to the educational goals and visions of individual medical schools in Korea. Furthermore, this measurement tool can serve as primary data for developing character qualities tools tailored to each medical school’s vision and educational goals.

Measuring Learning Loss Due to Disruptions from COVID-19: Perspectives from the Concept of Fractions

Ineffective learning during Covid-19 can be a major cause of learning loss. Through polytomous item response theory analysis, this study seeks to identify learning loss of fractions and investigate the reliability and validity of the DINA model diagnostic test. This study involved 177 Grade 8 students from six junior high schools in Banda Aceh, Indonesia, to obtain information about the validity and reliability of the diagnostic test and learning loss of fraction due to Covid-19. The research was conducted by designing diagnostic test, expert validation, and empirical testing. Multiple choice problems with polytomous options were developed. This study produced 20 valid problems based on the level of item discriminant, item difficulty, and slip and guess, with a reliability of 0.899. This study revealed that learning loss was experienced by 62 students from low-level junior high schools (100%), 51 students from middle-level junior high schools (98%), and 54 students from high-level junior high schools (85.7%). It can be said that low-level schools have the highest percentage of students with learning loss. It is recommended that future studies evaluate the effectiveness of diagnostic tests in identifying learning weaknesses in mathematics before semester exams using the items generated in this study.

The effect of scoring correction and model fit on the estimation of ability parameter and person fit on polytomous item response theory

Scoring quality has been recognized as one of the important aspects that should be of concern to both test developers and users. This study aimed to investigate the effect of scoring correction and model fit on the estimation of ability parameters and person fit in the polytomous item response theory. The result of 165 students in the Statistics course (SATS4410) test at one of the universities in Indonesia was used to answer the problems in this study. The polytomous data obtained from scoring the test results were analyzed using the Item Response Theory (IRT) approach with the Partial Credit Model (PCM), Graded Response Model (GRM), and Generalized Partial Credit Model (GPCM). The effect of scoring correction and model fit on the estimation of ability and person fit was tested using multivariate analysis. Among the three models used, GRM showed the best fit based on p-value and RSMEA. The results of the analysis also showed that there was no significant effect of scoring correction and model fit on the estimation of the test taker’s ability and person fit. From the results of this study, we recommend the importance of evaluating the levels or categories used in scoring student work on a test.

Detecting Rating Scale Malfunctioning With the Partial Credit Model and Generalized Partial Credit Model.

Rating scale analysis techniques provide researchers with practical tools for examining the degree to which ordinal rating scales (e.g., Likert-type scales or performance assessment rating scales) function in psychometrically useful ways. When rating scales function as expected, researchers can interpret ratings in the intended direction (i.e., lower ratings mean "less" of a construct than higher ratings), distinguish between categories in the scale (i.e., each category reflects a unique level of the construct), and compare ratings across elements of the measurement instrument, such as individual items. Although researchers have used these techniques in a variety of contexts, studies are limited that systematically explore their sensitivity to problematic rating scale characteristics (i.e., "rating scale malfunctioning"). I used a real data analysis and a simulation study to systematically explore the sensitivity of rating scale analysis techniques based on two popular polytomous item response theory (IRT) models: the partial credit model (PCM) and the generalized partial credit model (GPCM). Overall, results indicated that both models provide valuable information about rating scale threshold ordering and precision that can help researchers understand how their rating scales are functioning and identify areas for further investigation or revision. However, there were some differences between models in their sensitivity to rating scale malfunctioning in certain conditions. Implications for research and practice are discussed.

Excellence bias related to rating scales with summative jury assessment

Juries are a high-stake practice in higher education to assess complex competencies. However common, research remains behind in detailing the psychometric qualities of juries, especially when using rubrics or rating scales as an assessment tool. In this study, I analyze a case of a jury assessment (N = 191) of product development where both internal teaching staff and external judges assess and fill in an analytic rating scale. Using polytomous item response theory (IRT) analysis developed for the analysis of heterogeneous juries (i.e. jury response theory or JRT), this study attempts to provide insight into the validity and reliability of the used assessment tool. The results indicate that JRT helps detect unreliable response patterns that indicate an excellence bias, i.e. a tendency not to score in the highest response category. This article concludes with a discussion on how to counter such bias when using rating scales or rubrics for summative assessment.

Re-assessing the Psychometric Properties of Stress Appraisal Measure in Ghana Using Multidimensional Graded Response Model.

Despite the widespread use of the stress appraisal measure questionnaire in sport psychology literature, information on the psychometric properties of this survey instrument across different cultures and samples is still lacking. This study sought to validate the stress appraisal measure among male football players in the Ghana’s Premier League using the multidimensional item response theory. The descriptive cross-sectional survey design was adopted to recruit 424 footballers from the 2020/2021 Ghana Premier League season using the census approach. The 28-item Stress Appraisal Measure was used to assess six (6) appraisal mechanisms under primary and secondary cognitive appraisals. The ordered polytomous item response theory was used for analyzing the data. The study found that although some items were problematic, the majority of them were found to have good item parameters, effective scale option functioning, and provided adequate empirical information in the measurement of stress appraisal. This research concluded that the stress appraisal measure has promising applicability among male footballers who participated in the premier league in Ghana. Future researchers are encouraged to re-validate the stress appraisal measure with a different sample to contribute to the understanding of the applicability of the instrument in non-western populations.

Psychometric Validation of the Spanish Gaming Disorder Test (GDT): Item Response Theory and Measurement Invariance Analysis

Gaming Disorder (GD) has been recently added to the eleventh revision of the International Classification of Diseases (ICD-11) by the World Health Organization (WHO), as such, psychometrically sound psychological measures are required to assess this disorder. The objective of this study was to investigate the psychometric properties of the Spanish Gaming Disorder Test (GDT) by assessing its dimensionality, reliability, convergent validity, and associations with other variables of importance through polytomous Item Response Theory (IRT) and Measurement Invariance (MI) analysis across genders. To achieve this, a sample of 538 gamers (42.94% female, meanage = 23.29 years, SD = 7.24) was recruited. The results obtained supported a one-factor structure for the Spanish GDT with adequate reliability and convergent validity. Furthermore, satisfactory goodness of fit in the partial credit model (PCM) with more precise scores at high trait levels to assess GD was found, and strict invariance across genders was supported. These findings attest to the suitability of the Spanish GDT for clinical assessment and research on disordered gaming beyond community samples.

Performance of Polytomous IRT Models With Rating Scale Data: An Investigation Over Sample Size, Instrument Length, and Missing Data

The implementation of polytomous item response theory (IRT) models such as the graded response model (GRM) and the generalized partial credit model (GPCM) to inform instrument design and validation has been increasing across social and educational contexts where rating scales are usually used. The performance of such models has not been fully investigated and compared across conditions with common survey-specific characteristics such as short test length, small sample size, and data missingness. The purpose of the current simulation study is to inform the literature and guide the implementation of GRM and GPCM under these conditions. For item parameter estimations, results suggest a sample size of at least 300 and/or an instrument length of at least five items for both models. The performance of GPCM is stable across instrument lengths while that of GRM improves notably as the instrument length increases. For person parameters, GRM reveals more accurate estimates when the proportion of missing data is small, whereas GPCM is favored in the presence of a large amount of missingness. Further, it is not recommended to compare GRM and GPCM based on test information. Relative model fit indices (AIC, BIC, LL) might not be powerful when the sample size is less than 300 and the length is less than 5. Synthesis of the patterns of the results, as well as recommendations for the implementation of polytomous IRT models, are presented and discussed.

Bayesian estimation of multidimensional polytomous item response theory models with Q-matrices using Stan

The Q-matrix is commonly used in diagnostic classification models and has recently been incorporated into the multidimensional item response theory (MIRT) models to add information about the relationship between items and dimensions of the latent trait. The reformulation of the MIRT models with Q-matrix (MIRT-Q) has presented to improve the precision of the parameters of these models and to provide a simple and intuitive method for users to define the item-trait relationship. This paper aims to explore the incorporation of the Q-matrix in the formulation of MIRT models for polytomous item responses. Specifically, we introduce the incorporation of the Q-matrix into two of the polytomous MIRT models most known and used: the multidimensional graded response (MGR) model, hereinafter called MGR-Q, and the multidimensional generalized partial credit (MGPC) model, hereinafter called MGPC-Q. We provide readers the code of the MGR-Q and MGPC-Q models in Stan, a Bayesian estimation software, and we conduct a simulation study in order to evaluate the parameter recovery of the estimation method. To illustrate the use of both models in practice, we fit them to an operational dataset from 2400 individuals on 13 items and demonstrate the estimation of MGR-Q and MGPC-Q using the Stan program.

Assessing Cognitive Abilities Using the WAIS-IV: An Item Response Theory Approach.

Background: The Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) has been adapted to 28 different cultures and there has been considerable interest in examining its structure through exploratory and confirmatory factor analysis. This study investigates item and scale properties of the Egyptian WAIS-IV using item response theory (IRT) models. Methods: The sample consisted of 250 adults from Egypt. The item-level and subtest statistical properties of the Egyptian WAIS-IV were established using a combination of four dichotomous IRT models and four polytomous IRT models. In addition, factor analysis was performed to investigate the dimensionality of each subtest. Results: Factor analysis indicated the unidimensionality of each subtest. Among IRT models, the two-parameter logistic model provided a good fit for dichotomous subtests, while the graded response model fitted the polytomous data. Most items of the Egyptian WAIS-IV showed high discrimination, and the scale was adequately informative across the levels of latent traits (i.e., cognitive variables). However, each subtest included at least some items with limited ability to distinguish between individuals with differing levels of the cognitive variable being measured. Furthermore, most subtests have items that do not follow the difficulty rank they are ascribed in the WAIS-IV manual. Conclusions: Overall, the results suggest that the Egyptian WAIS-IV offers a highly valid assessment of intellectual abilities, despite the need for some improvements.

Validating the teaching, learning, and assessment quality of Malaysian ECCE instrument

<span>Several instruments that measure the teaching, learning, and assessment quality have been developed and published. However, a psychometrically sound instrument to measure teaching, learning, and assessment quality in early childhood care and education that suitable for the Malaysian context needs to be validated. Therefore, this study aimed to validate on teaching, learning, and assessment quality in early childhood care and education instrument, which contains 68 items. The sample comprised 3,498, selected by stratified random sampling from a population of all Malaysian kindergarten teachers. Data were analyzed based on the Polytomous Item Response Theory (IRT) using the Xcalibre software. Samejima's Graded Rating Model (SRGM) was found to be the fit model with the data. Unidimensionality assumption and local independence were tested using the exploratory factor analysis and were fulfilled. The instrument’s reliability was overall very good (α=0.966) and the construct validity was also fairly fulfilled with the value of 58.17% total variance explained. Therefore, this instrument is suggested to be used as fairly to measure the quality of Malaysian early childhood care and education among teachers so that appropriate follow-up actions can be implemented towards the betterment of early childhood education quality.</span>

Item response theory as a feature selection and interpretation tool in the context of machine learning.

Optimizing the number and utility of features to use in a classification analysis has been the subject of many research studies. Most current models use end-classifications as part of the feature reduction process, leading to circularity in the methodology. The approach demonstrated in the present research uses item response theory (IRT) to select features independent of the end-classification results without the biased accuracies that this circularity engenders. Dichotomous and polytomous IRT models were used to analyze 30 histological breast cancer features from 569 patients using the Wisconsin Diagnostic Breast Cancer data set. Based on their characteristics, three features were selected for use in a machine learning classifier. For comparison purposes, two machine learning-based feature selection protocols were run-recursive feature elimination (RFE) and ridge regression-and the three features selected from these analyses were also used in the subsequent learning classifier. Classification results demonstrated that all three selection processes performed comparably. The non-biased nature of the IRT protocol and information provided about the specific characteristics of the features as to why they are of use in classification help to shed light on understanding which attributes of features make them suitable for use in a machine learning context.