IRT Analysis Research Articles

Comparing age of cystic fibrosis diagnosis and treatment initiation after newborn screening with two common strategies

BackgroundNewborn screening (NBS) for CF has become widespread, although there are multiple strategies. Little is known about outcomes such as age of diagnosis after different NBS methods. MethodsWe used the U.S. Cystic Fibrosis Foundation Patient Registry to identify infants with CF born between 2001 and 2008 in states that utilized NBS. We compared ages at diagnosis, genotyping, sweat test, and first visit to a CF Centre between states that used serial immunoreactive trypsinogen (IRT/IRT) levels and states that used IRT and DNA analysis (IRT/DNA). ResultsWe identified 1288 infants with CF. Compared to infants born in IRT/IRT states, infants born in IRT/DNA states were younger at the time of diagnosis (median 2.3weeks versus 4.0weeks in IRT/IRT states, p<0.001), genotyping (0.7weeks versus 5.3weeks, p<0.001), and initial CF Centre visit (5.9weeks versus 7.7weeks, p=0.008). ConclusionsAlthough there is room to improve outcomes with both strategies, infants born in IRT/DNA states have treatment initiated at a younger age than infants born in IRT/IRT states.

MEASURING SCIENCE INTEREST: RASCH VALIDATION OF THE SCIENCE INTEREST SURVEY

Students in the USA have fallen near the bottom in international competitions and tests in mathematics and science. It is thought that extrinsic factors such as family, community, and schools might be more influential than intrinsic attitudes toward science interest. However, there are relatively few valid and reliable measures of intrinsic factors such as interest relating to science. With the lack of intrinsic measures, it is difficult to determine the impact of extrinsic factors on the intrinsic construct. A fuller picture of the factors affecting intrinsic factors such as science interest will allow interventions to become more refined and targeted. Several studies suggest that student interest toward science affects the likelihood of the student pursuing advanced courses in science. The goal of this paper is to establish the validity and reliability of the Science Interest Survey and to determine if the survey meets the formal requirements of measurements as defined by the Rasch model. Results using both IRT and CRT analysis suggest that Science Interest Survey is an adequate measure of the unidimensional construct known as science interest. Results further suggest the Science Interest Survey is a valid and reliable measure for assessing science interest levels.

A Mixture IRT Analysis of Risky Youth Behavior

The study reported in this manuscript used a mixture item response model with data from the Youth Risk Behavior Survey 2009 (N = 16,410) to identify subtypes of adolescents at-risk for engaging in unhealthy behaviors, and to find individual survey items that were most effective at identifying such students within each subtype. The goal of the manuscript is twofold: (1) To demonstrate the utility of the mixture item response theory model for identifying subgroups in the population and for highlighting the use of group specific item response parameters and (2) To identify typologies of adolescents based on their propensity for engaging in sexually and substance use risky behaviors. Results indicate that four classes of youth exist in the population, with differences in risky sexual behaviors and substance use. The first group had a greater propensity to engage in risky sexual behavior, while group 2 was more likely to smoke tobacco and drink alcohol. Group 3 was the most likely to use other substances, such as marijuana, methamphetamine, and other mind altering drugs, and group 4 had the lowest propensity for engaging in any of the sexual or substance use behaviors included in the survey. Finally, individual items were identified for each group that can be most effective at identifying individuals at greatest risk. Further proposed directions of research and the contribution of this analysis to the existing literature are discussed.

Assessing the factor structure of a role functioning item bank

Role functioning (RF) is an important part of health-related quality of life, but is hard to measure due to the wide definition of roles and fluctuations in role participation. This study aims to explore the dimensionality of a newly developed item bank assessing the impact of health on RF. A battery of measures with skip patterns including the new RF bank was completed by 2,500 participants answering only questions on social roles relevant to them. Confirmatory factor analyses were conducted for the participants answering items from all conceptual domains (N=1193). Conceptually based dimensionality and method effects reflecting positively and negatively worded items were explored in a series of models. A bi-factor model (CFI=.93, RMSEA=.08) with one general and four conceptual factors (social, family, occupation, generic) was retained. Positively worded items were excluded from the final solution due to misfit. While a single factor model with methods factors had a poor fit (CFI=.88, RMSEA=.13), high loadings on the general factor in the bi-factor model suggest that the RF bank is sufficiently unidimensional for IRT analysis. The bank demonstrated sufficient unidimensionality for IRT-based calibration of all the items on a common metric and development of a computerized adaptive test.

Multilevel Mixed-Measurement IRT Analysis: An Explication and Application to Self-Reported Emotions Across the World

Dimensional approaches assume that all individuals within hierarchical units (e.g., organizations or countries) share the same measurement model. However, such models are less applicable when researchers are interested in obtaining classes of individuals who share the same measurement model across hierarchical units and to obtain hierarchical latent classes (LCs). The authors present the multilevel mixed-measurement item response theory (MMM-IRT) model as an alternative. This model yields classes of individuals with a common measurement model that spans across hierarchical units. In addition, hierarchical units are classified together to the extent that they share similar proportions of individual-level classes. The authors illustrate the MMM-IRT model with data on self-reported emotions from 121,740 individuals across 116 countries where four individual classes and five country classes were found. Theoretical and methodological implications concerning cross-cultural, multilevel, and measurement equivalence research are discussed.

An item response theory analysis of Wong and Law emotional intelligence scale

The purpose of this study was to perform an IRT analysis of Wong and Law Emotional Intelligence Scale (WLEIS: Wong & Law, 2002). The sample consisted of 481university students in the province of Balochistan, Pakistan. After examining the unidimensionality of WLEIS four sub-scales, graded response model for seven ordered categories was applied (Samejima, 1969) using “ltm” an R package (Rizopoulous, 2006). Results indicated that the WLEIS sub-scales yield precise measurement for individuals with low to moderate trait levels and relatively imprecise measurement for individuals with high trait levels.

Development of IRT analysis programs for beginners : EasyEstimation series

Development of IRT analysis programs for beginners : EasyEstimation series

A psychometric analysis of functional category production in English agrammatic narratives

Hierarchical models of agrammatism propose that sentence production deficits can be accounted for in terms of clausal syntactic structure ( Friedmann & Grodzinsky, 1997; Hagiwara, 1995). Such theories predict that morpho-syntactic elements associated with higher nodes in the syntactic tree (complementizers and verb inflections) will be more impaired than elements associated with lower structural positions (negation markers and aspectual verb forms). While this hypothesis has been supported by the results of several studies ( Friedmann & Grodzinsky, 1997; Hagiwara, 1995), it has also been challenged on several grounds ( Burchert, Swoboda-Moll, & De Bleser, 2005a; Lee, 2003; Lee, Milman, & Thompson, 2005). In this paper the question of hierarchical structure was re-examined within the framework of Item Response Theory (IRT, Rasch, 1980). IRT is a probabilistic model widely used in the field of psychometrics to model behavioral constructs as numeric variables. In this study we examined production of functional categories (complementizers, verb inflections, negation markers, and aspectual verb forms) in narrative samples elicited from 18 individuals diagnosed with nonfluent aphasia and 18 matched controls. Data from the aphasic participants were entered into an IRT analysis to test (1) whether production of clausal functional categories can be represented as a variable on a numeric scale; and (2) whether production patterns were consistent with hierarchical syntactic structure. Pearson r correlation coefficients were also computed to determine whether there was a relation between functional category production and other indices of language performance. Results indicate that functional category production can be modeled as a numeric variable using IRT. Furthermore, although variability was observed across individuals, consistent patterns were evident when the data were interpreted within a probabilistic framework. Although functional category production was moderately correlated with a second measure of clausal structure (clause length), it was not correlated with more distant language constructs (noun/verb ratio and WAB A.Q.). These results suggest that functional category production is related to some, but not all, measures of agrammatic language performance.

Nonparametric IRT analysis of Quality-of-Life Scales and its application to the World Health Organization Quality-of-Life Scale (WHOQOL-Bref)

BackgroundThis study investigates the usefulness of the nonparametric monotone homogeneity model for evaluating and constructing Health-Related Quality-of-Life Scales consisting of polytomous items, and compares it to the often-used parametric graded response model.MethodsThe nonparametric monotone homogeneity model is a general model of which all known parametric models for polytomous items are special cases. Merits, drawbacks, and possibilities of nonparametric and parametric models and available software are discussed. Particular attention is given to the monotone homogeneity model (also known as the Mokken model), and the often-used parametric graded response model.ResultsData from the WHOQOL-Bref were analyzed using both the monotone homogeneity model and the graded response model. The monotone homogeneity model analysis yielded unidimensional scales for each content domain. Scalability coefficients further showed that some items have limited scalability with respect to the other items in the same scale. The parametric IRT analyses lead to the rejection of some of the items.ConclusionsThe nonparametric monotone homogeneity model is highly suited for data analysis in a health-related quality-of-life context, and the parametric graded response model may add interesting features to measurement provided the model fits the data well.

The measurement of visuo–spatial and verbal–numerical working memory: Development of IRT-based scales

The objective of this study is to explore the potential for developing IRT-based working memory scales for assessing specific working memory components in children (8–13 years). These working memory scales should measure cognitive abilities reliably in the upper range of ability distribution as well as in the normal range, and provide a much-needed, reliable, and valid test for assessing high intellectual abilities in children. Six computer-assisted working memory tasks were administered to 172 children from regular schools and to 202 children from special schools and other institutions for the gifted. A factor analysis revealed a two-factor structure and the existence of a verbal–numerical and a visuo–spatial working memory scale. Classical item analysis and IRT analysis yielded good psychometric properties for both scales and revealed that the scales are appropriate for measuring high cognitive abilities. Both scales showed substantial and differential power for the explanation of variance in school achievement.

Evaluating the effectiveness of instructional resource allocation and use: IRT and HLM analysis of NAEP teacher survey and student assessment data

Evaluating the effectiveness of instructional resource allocation and use: IRT and HLM analysis of NAEP teacher survey and student assessment data

Application de la théorie de réponse aux items à l'analyse d'échelles d'attitude

Application of Item Response Theory to the Analysis of Attitude Scales. The article presents Item Response Theory (lRT) and illustrates its application to the analysis of two measures of work-related attitudes -- value given to social aspects of the job and satisfaction with extrinasic aspects of the job. Results of IRT analysis show that extrinsic satisfaction cannot be measured in the same way among contract professionals as opposed to regular employees. Using information provided by IRT analysis. it appeared that satisfaction with salary cannot be included in a scale measuring extrinsic satisfaction because it does not contribute to the measure in the same way for the two groups of employees: furthermore, satisfaction with social benefits may be included in the scale only if discrimination and difficulty parameters are allowed to vary across groups. Analyses of variance using theta scores estimated in this way give statistical conclusions that are notably different form those obtained using traditional additive scores and permit refinement of interpretation.

A Partial Test of Hulin's Psychometric Theory of Measurement Equivalence in Translated Tests

Hulin (1987) suggested that the a and b parameters of item characteristic curves (ICCs) for test items that function differentially when translated for use in another culture (i.e., DIF items) may provide useful information about the source of DIF. Hulin hypothesized that a-parameter differences and b-parameter differences “have much different linguistic and measurement implications.” He proposed that DIF due to b-parameter differences is due to translation error; DIF due to a-parameter differences is indicative of more “profound cultural bias.” The 118-item Trier Personality Inventory was administered (in German) shortly after reunification to 300 East and 298 West Germans. Thus, the source of any DIF discovered could not be due to translation error. IRT analysis revealed two significant DIF items and two items with DIF indices that approached significance. In disagreement with Hulin, there were large differences in the b parameters along with modest differences in the a parameters for three out of four DIF items. Likewise, examination of ICCs for DIF items failed to provide empirical support for Hulin's hypothesis. Implications of these findings for test translators are discussed.

An IRT Investigation of the Validity of Non-Patient Analogue Research Using the Beck Depression Inventory

This paper presents an IRT analysis of the Beck Depression Inventory which was carried out to assess the assumption of an underlying latent trait common to non-clinical and patient samples. A one parameter rating scale model was fitted to data drawn from a patient and non-patient sample. Findings suggest that while the BDI fits the model reasonably well for the two samples separately there is sufficient differential item functioning to raise serious duobts of the viability of using it analogously with patient and non-patient groups.

The role of test-taker feedback in the test development process: test-takers' reactions to a tape-mediated test of proficiency in spoken Japanese

This study explores how test-taker reactions in a specific purpose testing context may vary according to characteristics of the test-taker. Such reactions are of interest both theoretically to the researcher and practically to the test developer, who is concerned to ensure that the test is fair and appropriate for all candidates and acceptable to the range of test-takers. The reactions may also be of use in the improvement of test items (as a supplement to item analysis) and test rubrics. The article reports on the use of test-taker feedback in the development of the occupational foreign language test (Japanese), a tape-mediated test of spoken Japanese for the tourism and hospitality industry. Some 53 trial subjects completed a post-test questionnaire, providing reactions to the test as a whole, to task types, and to individual test items. Relationships are investigated between responses and a number of test-taker characteristics, including gender, type of course undertaken (general or specific purpose), amount of study of the language, time spent in Japan and relevant occupa tional experience. Responses are also considered for items and person shown by Rasch IRT analysis as indicating significant misfit. Aspects of the content, construct and face validity of the test are considered in the light of the analysis. The role of feedback in the revision of test items, in the writing of test rubrics and in the development of the test-user handbook is also discussed.

The Use of Item Statistics in the Calibration of an Item Bank

An IRT analysis based on p (proportion correct) and r (item-test correlation) is proposed for a group of tests having items in common. The procedure is a generalization of a procedure proposed by De Gruijter and Mooijaart (1983) which is related to procedures for the factor analysis of dichotomous data. The pro cedure results in IRT item parameters using data from examinee groups with subsets of common items; it is, therefore, particularly appropriate for calibrating items for use in small-scale item banks. Simulated data are used to illustrate the procedure.

Some serial properties of burst responding on DRL

A sequential IRT analysis was carried out on the response output of two rats on DRL 15 sec LH 5 sec. It was found that burst responding (1) did not occur after a reinforced response, (2) was more likely to follow a nonreinforced response if the latter was preceded by a reinforced response, (3) increased in probability after a nonreinforced response when the prior run of reinforced responses was greater than one, and (4) decreased in probability as the last successfully timed response became more remote. As reinforcement on this schedule is known to initiate a more precise temporal discrimination, it was hypothesized that these results might explain a previously reported finding that burst responding increases in probability as an IRT approaches the lower bound of the reinforcement interval.