Item Information Functions Research Articles

Increased energy/activity, not mood changes, is the core feature of mania

BackgroundIn the Diagnostic and Statistical Manual of Mental Disorders, 5th edition, the occurrence of increased energy/activity and elation of mood or irritability became necessary symptoms for the diagnosis of an episode of mania or hypomania. ObjectiveTo evaluate whether increases in energy/activity or mood changes represent the core feature of the manic syndrome. MethodsThe symptomatology of 117 hospitalized patients with bipolar mania was evaluated using the Schedule for Affective Disorders and Schizophrenia-Changed version (SADS-C). Based on six items of the SADS-S related to mania, a Confirmatory Factor Analysis (CFA) was performed. An Item Response Theory (IRT) analysis was used to identify how much each symptom informs about the different levels of severity of the syndrome. ResultsAccording to the CFA, the item “increased energy” was the symptom with the highest factorial loadings, which was confirmed by the IRT analysis. Thus, increased energy was the alteration most correlated with the total severity of manic symptoms. Additionally, the analysis of the Item Information Function revealed that increased energy was correlated with the larger amplitude of severity levels compared with the other symptoms of mania. LimitationsOnly six manic symptoms were considered. The sample might not be representative because the patients were evaluated while presenting peak symptom severity. ConclusionsIncreased energy/activity is a more important symptom for a diagnosis of mania than mood changes and represents the core feature of this syndrome.

Identification of Developmentally Appropriate Screening Items for Disruptive Behavior Problems in Preschoolers

Screening preschool-aged children for disruptive behavior disorders is a key step in early intervention. The study goal was to identify screening items with excellent measurement properties at sub-clinical to clinical levels of disruptive behavior problems within the developmental context of preschool-aged children. Parents/caregivers of preschool-aged children (N = 900) were recruited from four pediatric primary care settings. Participants (mean age = 31, SD = 8) were predominantly female (87%), either white (55%) or African-American (42%), and biological parents (88%) of the target children. In this cross-sectional survey, participants completed a sociodemographic questionnaire and two parent-report behavioral rating scales: the PSC-17 and the BPI. Item response theory analyses provided item parameter estimates and information functions for 18 externalizing subscale items, revealing their quality of measurement along the continuum of disruptive behaviors in preschool-aged children. Of 18 investigated items, 5 items measured only low levels of disruptive behaviors among preschool-aged children. The remaining 13 items measured sub-clinical to clinical levels of disruptive behavior problems (i.e., >1.5 SD); however, 5 of these items offered less information, suggesting unreliable measurement. The remaining 8 items had high discrimination and difficulty parameters, offering considerable measurement information at sub-clinical to clinical levels of disruptive behavior problems. Behaviors measured by the 8 selected parent-report items were consistent with those identified in recent efforts to distinguish developmentally typical misbehaviors from clinically concerning behaviors among preschool-aged children. These items may have clinical utility in screening young children for disruptive behavior disorders.

A Note on the Item Information Function of the Four-Parameter Logistic Model

This article focuses on four-parameter logistic (4PL) model as an extension of the usual three-parameter logistic (3PL) model with an upper asymptote possibly different from 1. For a given item with fixed item parameters, Lord derived the value of the latent ability level that maximizes the item information function under the 3PL model. The purpose of this article is to extend this result to the 4PL model. A generic and algebraic method is developed for that purpose. The result is practically illustrated by an example and several potential applications of this result are outlined.

Multidimensional Grade Response Model

Multidimensional Item Response Theory(MIRT),which is based on factor analysis and unidimensional Item Response Theory(IRT),is one of a new development trend of IRT.It's a fact that MIRT is in an early-developing stage and most studies are mainly concentrated on MIRT models for items with two score categories.With respect to polytomous MIRT models,it's until 1993 that Muraki and Carlson produced a generalization of unidimensional Grade Response Model(GRM) and it uses response functions that have the normal ogive form.Some other models such as multidimensional Generalized Partial Credit Model(MGPCM) and Continuous Response Model(MCRM) are even developed in recent years(Yao Schwarz,2006;Ferrando,2009).In the paper,a form of logistic Multidimensional Graded Response Model(MGRM) is firstly presented.The graphics,which are plotted by Matlab 2007,and properties for a special case of two-dimensional GRM are demonstrated.Then,base on the definition of item information for a dichotomous MIRT models,the item information function for MGRM is derived and item information for a case of two-dimensional GRM discussed.Moreover,the main ideas of Joint Maximum Likelihood Estimation(JML) and Markov Chain Monte Carlo(MCMC) methods to estimate MGRM parameters are stated.Finally,some significant further researches,which include research of item and test information,developing parameter estimate program for MGRM,are illustrated in the paper.

The Nursing Student Self-Efficacy Scale

For nurse educators to optimize instruction that supports students' accurate estimates of their ability, a sound measure of nursing self-efficacy is needed. To provide evidence for an adequate item response theory model fit to scores measuring nursing students' self-efficacy to care for critically ill patients and to use the theory to provide evidence for reliable and valid interpretation of self-efficacy scores. The Nursing Student Self-Efficacy Scale was administered to 421 nursing students. Data regarding students' self-efficacy beliefs for psychomotor skill performance and communication skills were analyzed using the generalized partial credit item response theory model. All items showed acceptable model-data fit using the chi-square test with an alpha of .001. Review of item information functions showed that 40% of the items provided high information about self-efficacy and 20% provided moderately high information. Comparison of trait level distributions with reliability information given by the two subscales showed that the psychomotor subscale measured self-efficacy most accurately for second and third semester students and the communication subscale measured efficacy most accurately for first and second semester students. Significant mean differences in self-efficacy scores from both subscales were found between students in successive semesters of the program. Data analysis provided some evidence for reliable interpretation of scores indicating self-efficacy beliefs of nursing students. Comparison of scores from students in different semesters provided validity evidence; the instrument could be used to discriminate between beginning students and more advanced students possessing different levels of self-efficacy. When used for scale development, item response theory procedures can be more informative about item and test reliability than classical true score theory.

New Item Selection Criteria of Computerized Adaptive Testing with Exposure-Control Factor

As far as Computerized Adaptive Testing (CAT) is concerned, the issue of item selection strategy has received more attention because of its vital role. It is well known that there are two typical selection strategies called Maximum Information Criterion (MIC) and a-Stratification (a-STR). However, both of the two strategies have their advantages together with their downsides. On the one hand, MIC method can obtain high efficiency and accurate estimation of ability; on the other hand, its uneven item selection may lead to the insecurity of examination. Meanwhile, though a-STR can improve the test security by controlling the item exposure rate, it may result in the inefficiency of the test and failure in adjusting the discrimination within the layers. As a result, the development of both effective and safe item selection strategies has always been a goal to pursue in studies on CAT. According to the previous studies, the test security can be enhanced and the item pool utilization rate can be increased by balancing the item exposure rate. Therefore, in 0-1 scored CAT, two new item selection strategies are proposed in this paper to improve the MIC and a-STR methods by introducing exposure factor, adjusting automatically the discrimination by stage and increasing the accuracy of item selection. One of the new item selection strategies has three prominent characteristics: First, a function of item information (FII) rather than the item information function is set up to combine the advantages of both MIC and a-STR. Second, the effect of the discrimination on different stages in CAT is taken into account and a function of item discrimination is used in the FII to make up for the defect of a-STR for not being able to control the item discrimination in the internal layer. Third, mechanism of online control exposure is adopted. While some specific items in a certain examination process are more frequently exposured than others, with the help of the mechanism, they will turn out to become less likely to be selected in the future tests. At the same time, some specific items in a certain examination process are less frequently exposured than others, with the help of the mechanism, they will turn out to become more likely to be selected in the future tests. Thanks to the mechanism of online control exposure, the whole exposure rate of all the items in the item pool is evened and the utilization rate of the item pool is increased. In order to fill the gap between the exposure rate of each item and mean exposure rate of all the items in the item pool, this paper attempts to treat the item exposure rate directly as a part of the selection strategies expression. And it also tries to equalize the exposure rate by decreasing the exposure rate of the items with high ones and increasing the uses of the items with low ones. The approach differs from the approaches which only control the items with high exposure rate, e.g. SH. The results of Monte Carlo simulations show that compared with other approaches, the approach proposed in this paper is more effective in terms of exposure control and more ideal in the performance of other indexes.

Item Response Theory Analyses of Adult Self-Ratings of the ADHD Symptoms in the Current Symptoms Scale

The graded response model, which is based on item response theory, was used to evaluate the psychometric properties of adult self-ratings (N = 852) of the attention deficit/hyperactivity disorder inattention, hyperactivity, and impulsivity symptoms presented in the Current Symptoms Scale. This scale has four ordered response categories. The results for the discrimination parameters showed that all symptoms were generally good for discriminating their respective latent traits. For virtually all symptoms, their threshold values showed that they were especially good at representing the appropriate traits from around the mean trait level onward. The item information function values for most symptoms indicated reasonable reliability from approximately the mean trait level onward. All these findings are new and extend existing psychometric information for adult self-ratings of the attention deficit/hyperactivity disorder symptoms in the Current Symptoms Scale.

Interpretation of the Three-Parameter Testlet Response Model and Information Function

The testlet response model is designed for handling items that are clustered, such as those embedded within the same reading passage. Although the testlet is a powerful tool for handling item clusters in educational and psychological testing, the interpretations of its item parameters, the conditional correlation between item pairs, and the testlet information function are not always clear. For example, the a-parameter in a testlet response model, which is sometimes referred to as the discrimination parameter and interpreted in the conventional way, is often directly compared to the a-parameter in standard item response models. In this article, it is argued that despite the deceptively similar forms of the a-parameters, such interpretation and comparison can be misleading. A marginalized version of the testlet model is introduced to derive quantities that are comparable to standard item response models. This mean (marginalization) of the function approach is contrasted with the temptingly simple but inappropriate function of the mean approach, in which the testlet effect is set to its mean of zero in respective equations for item response, correlation, and information functions. Testlet data collected from a science assessment within the National Assessment of Educational Progress are used to illustrate the subtle differences in the interpretations of the various response models.

A Comparison of Content-Balancing Procedures for Estimating Multiple Clinical Domains in Computerized Adaptive Testing: Relative Precision, Validity, and Detection of Persons With Misfitting Responses.

This simulation study sought to compare four different computerized adaptive testing (CAT) content balancing procedures designed for use in a multidimensional assessment with respect to measurement precision, symptom severity classification; validity of clinical diagnostic recommendations; and sensitivity to atypical responding. The four content balancing procedures were: (1) no content balancing, (2) Screener-based, (3) Mixed (Screener plus content balancing), and (4) Full content balancing. In Full content balancing and in Mixed content balancing following administration of the screener items, item selection was based on: (1) whether the target number of items for the item's subscale was reached, and (2) the item's information function. Mixed and Full content balancing provided the best representation of items from each of the main IMDS subscales. These procedures also resulted in higher CAT-to-full-scale correlations for the Trauma and Homicidal/Suicidal Thought subscales, and improved detection of atypical responding.

Differential Item Functioning Analysis Using Rasch Item Information Functions

Differential item functioning (DIF) analysis is a statistical technique used for ensuring the equity and fairness of educational assessments. This study formulates a new DIF analysis method using the information similarity index (ISI). ISI compares item information functions when data fits the Rasch model. Through simulations and an international assessment example, ISI, the signed area index (SAI), and Mantel-Haenszel procedure, are compared. Results show that in many circumstances, ISI performs comparably, if not better than MH and SAI in terms of its capacity to detect DIF. Additionally, guidelines for assessing practically and statistically significant DIF using ISI are provided.

Questioning the measurement precision of psychotherapy research

Recent developments in psychometric theory have important implications for the development and validation of the instruments used in psychotherapy research. This article illustrates how the advances in latent trait and item response theory (IRT) have distinct advantages over classical test theory. The basic assumptions of IRT (unidimensionality, local independence, and monotonicity) and the concepts common to all four of the basic IRT models (item characteristic curve, item information function, and test information function) are reviewed. IRT methods are illustrated using responses to a psychotherapy outcome instrument from a large data set of a commercial health plan.

IRT Information Function of Polytomously Scored Items under the Graded Response Model

Computerized adaptive testing(CAT) is one of the ultimate areas in the field of item response theory(IRT).Many high stake tests,such as GRE and TOEFL,have their CAT versions. Item selection strategy is the core content of CAT.And item information function(IIF) always is the important index of item selection.Although item information of dichotomously scored items has been extensively studied,item information of polytomously scored items receives much less attention. However,due to the advantages inbred in Computerized adaptive testing(CAT) with polytomously scored items,it gains more and more attention now.But the item selection strategies implemented under such situations are not systematically proved to be efficient.Many researchers use the degree of closeness between trait level and the average of item category parameters as the index of item selection strategy,or other strategies such as the degree of closeness between trait level and the median of item category parameters,etc. Up to now,seldom research had systematically concerned about the inherent relationship between the trait level and item category parameters under polytomously scored item types,and its effect on item information. The primary purpose of this research is to systematically investigate the relations of item information to item category parameters and subject trait levels. In this study,we simulated 121 trait values that distributed uniformly between the ranges of-3 to 3.Also,we simulated 504 sets of item parameters,with 4 sets of discrimination parameters which separately matched the 126 sets of difficulty parameters.Each item is graded in terms of 5 categories with differential degrees of difficulty. Based on the results of item information of simulated data,we find that the trait value that correspondence to the maximum item information matches the difficulty parameter group with high-frequency item categories.We call this principal as "item category parameter priority rule".Such principle is very different from the previous item selection strategies under computerized adaptive testing situations. The results of this research will be very useful for the construction of computerized adaptive testing with polytomously scored items.

The curiosity and exploration inventory-II: Development, factor structure, and psychometrics

Given curiosity’s fundamental role in motivation, learning, and well-being, we sought to refine the measurement of trait curiosity with an improved version of the Curiosity and Exploration Inventory (CEI; [Kashdan, T. B., Rose, P., & Fincham, F. D. (2004). Curiosity and exploration: Facilitating positive subjective experiences and personal growth opportunities. Journal of Personality Assessment, 82, 291–305]). A preliminary pool of 36 items was administered to 311 undergraduate students, who also completed measures of emotion, emotion regulation, personality, and well-being. Factor analyses indicated a two-factor model—motivation to seek out knowledge and new experiences ( Stretching; five items) and a willingness to embrace the novel, uncertain, and unpredictable nature of everyday life ( Embracing; five items). In two additional samples ( ns = 150 and 119), we cross-validated this factor structure and provided initial evidence for construct validity. This includes positive correlations with personal growth, openness to experience, autonomy, purpose in life, self-acceptance, psychological flexibility, positive affect, and positive social relations, among others. Applying item response theory (IRT) to these samples ( n = 578), we showed that the items have good discrimination and a desirable breadth of difficulty. The item information functions and test information function were centered near zero, indicating that the scale assesses the mid-range of the latent curiosity trait most reliably. The findings thus far provide good evidence for the psychometric properties of the 10-item CEI-II.

Item Analysis of C, D and E Series from Raven’s Standard Progressive Matrices with Item Response Theory Two-Parameter Logistic Model

The present report is focused on the Item Response Theory research methodology and descriptive potential. Its purpose is to present the item analysis of C, D and E Series from Raven’s Standard Progressive Matrices which were given to 506 Bulgarian high school students. The basic concepts and underlying assumptions of IRT are briefly reviewed. The latent variable is defined after a short check of some intelligence theories and a detailed examination of the used items. After the verification of IRT assumptions, the Two-parameter logistic model is selected for the analysis. The estimated item parameters are interpreted in accordance to suggested guidelines. Item Characteristic Curves and Item Information Functions are plotted and their features are also discussed.

Constructing optimal educational tests using GMDH-based item ranking and selection

Item ranking and selection plays a key role in constructing concise and informative educational tests. Traditional techniques based on the item response theory (IRT) have been used to automate this task, but they require model parameters to be determined a priori for each item and their application becomes more tedious with larger item banks. Machine-learning techniques can be used to build data-based models that relate the test result as output to the examinees’ responses to various test items as inputs. With this approach, test item selection can benefit from the vast amount of literature on feature selection in many areas of machine learning and artificial intelligence that are characterized by high data dimensionality. This paper describes a novel technique for item ranking and selection using abductive network pass/fail classifiers based on the group method of data handling (GMDH). Experiments were carried out on a dataset consisting of the response of 2000 examinees to 45 test items together with the examinee's true ability level. The approach utilizes the ability of GMDH-based learning algorithms to automatically select optimum input features from a set of available inputs. Rankings obtained by iteratively applying this procedure are similar to those based on the average item information function (IIF) at the pass-fail ability threshold, IIF ( θ=0), and the average information gain (IG). An optimum item subset derived from the GMDH-based ranking contains only one third of the test items and performs pass/fail classification with 91.2% accuracy on a 500-case evaluation subset, compared to 86.8% for a randomly selected item subset of the same size and 92% for a subset of the 15 items having the largest values for IIF ( θ=0). Item rankings obtained with the proposed approach compare favorably with those obtained using neural network modeling and popular filter type feature selection methods, and the proposed approach is much faster than wrapper methods employing genetic search.

Item Response Theory Analyses of the Parent and Teacher Ratings of the DSM-IV ADHD Rating Scale

The graded response model (GRM), which is based on item response theory (IRT), was used to evaluate the psychometric properties of the inattention and hyperactivity/impulsivity symptoms in an ADHD rating scale. To accomplish this, parents and teachers completed the DSM-IV ADHD Rating Scale (DARS; Gomez et al., Journal of Child Psychology and Psychiatry, 40, 265-274, 1999) for a group of 1,475 primary school-aged children. The results for the discrimination parameters showed that all symptoms for both groups of respondents were generally good for discriminating their respective latent traits. For virtually all symptoms, their threshold values showed moderate to large increases in the level of the latent trait at each subsequent response dichotomy, with the symptoms being especially good at representing the appropriate traits from mean to moderately high trait levels. The item information function values for most symptoms indicated reasonable reliability from, approximately, the mean trait levels to moderately high trait levels. These findings indicate good psychometric properties for the parent and teacher ratings of the DARS. The implications of the findings for the use of the DARS and other similar scales are discussed.

An IRT Analysis of the Personal Optimism Scale

Abstract. In this study the psychometric properties of the Personal Optimism scale of the POSO-E questionnaire ( Schweizer & Koch, 2001 ) for the assessment of dispositional optimism are evaluated by applying Samejima's (1969) graded response model, a parametric item response theory (IRT) model for polytomous data. Model fit is extensively evaluated via fit checks on the lower-order margins of the contingency table of observed and expected responses and visual checks of fit plots comparing observed and expected category response functions. The model proves appropriate for the data; a small amount of misfit is interpreted in terms of previous research using other measures for optimism. Item parameters and information functions show that optimism can be measured accurately, especially at moderately low to middle levels of the latent trait scale, and particularly by the negatively worded items.

An Application of Item Response Theory to the DSM-III-R Criteria for Borderline Personality Disorder

This paper summarizes results from analyses of the DSM criteria for borderline personality disorder (BPD) using models from item response theory (IRT). The study sample consisted of 353 participants, the majority of whom were psychiatric patients. Confirmatory factor analysis showed that a one-factor model provided the best fit to the data. All the DSM BPD criteria had moderate or higher item discrimination parameters, indicating that all items contributed meaningful information in assessing BPD. Item information functions revealed that the BPD criteria as a whole were useful for capturing BPD traits in the moderately severe to severe range, but that they performed less well in the less severe range. The general conclusion is that the criteria do represent a coherent syndrome and that further research on the informational value of the individual criteria would be useful.

Development and validation of the Generalized Anxiety Disorder Inventory (GADI)

The psychometric tools used for the assessment of generalized anxiety disorder (GAD) either do not conform to the current concept of the condition or have important limitations. We aimed to develop and validate a new questionnaire for the assessment of symptom profile and severity of GAD. An original pool of potential scale items was subjected to a series of studies in non-clinical and clinical populations, in order to determine the final composition of the scale. The psychometric properties of the new scale, the Generalized Anxiety Disorder Inventory (GADI), were evaluated using a factor analytic model suitable for ordinal data and the Graded Response Model. The precision of measurement of the GADI was quantified through the item information functions.A total of 197 outpatients and 522 non-clinical subjects participated in four studies and completed the GADI. The final 18-item scale was derived from an original pool of 30 potential items. The GADI showed good reliability, convergent and divergent validity. The scale comprises three factors, relating to cognitive, somatic and sleep symptoms. It accurately distinguished GAD patients from non-patient controls. The cognitive factor also distinguished GAD from other anxiety disorders and depression. The GADI is a useful tool in the assessment of the breadth of symptoms and the severity of generalized anxiety disorder in clinical settings.

Using Polytomous Item Response Models to Assess Death Anxiety

: The study of human attitudes toward death has given rise to a substantial body of empirical research. Psychometric instruments have been developed to measure fear of death, or death anxiety, and its psychological consequences in people who continually come into contact with stimuli related to mortality. : To analyze the 20-item Death Anxiety Inventory (DAI) within the framework of item response theory (IRT) and using the generalized partial credit model. : The sample comprised 154 men and 550 women and was drawn from nurses, doctors, industrial workers, teachers, undergraduates, and retired persons. Subjects completed the DAI, a self-administered, Likert-type questionnaire of 20 items, each with six response options. : The DAI showed a relatively adequate fit to the generalized partial credit model. Thus, 4 of the 20 items presented a poor fit to the model. The analysis of item information and test information functions revealed that the 20-item test was appropriate for differentiating subjects with medium or high levels of death anxiety. The test information function was higher in this range of scores, indicating greater precision in the estimate of death anxiety for these subjects. : The generalized partial credit model can be used to obtain detailed information about a clinical test and its items, and there are advantages to this approach when working with polytomous tests.