Two-parameter Item Response Model Research Articles

Type I error and power rates: A comparative analysis of techniques in differential item functioning

The main purpose of this study is to examine the Type I error and statistical power ratios of Differential Item Functioning (DIF) techniques based on different theories under different conditions. For this purpose, a simulation study was conducted by using Mantel-Haenszel (MH), Logistic Regression (LR), Lord’s χ2, and Raju’s Areas Measures techniques. In the simulation-based research model, the two-parameter item response model, group’s ability distribution, and DIF type were the fixed conditions while sample size (1800, 3000), rates of sample size (0.50, 1), test length (20, 80) and DIF- containing item rate (0, 0.05, 0.10) were manipulated conditions. The total number of conditions is 24 (2x2x2x3), and statistical analysis was performed in the R software. The current study found that the Type I error rates in all conditions were higher than the nominal error level. It was also demonstrated that MH had the highest error rate while Raju’s Areas Measures had the lowest error rate. Also, MH produced the highest statistical power rates. The analysis of the findings of Type 1 error and statistical power rates illustrated that techniques based on both of the theories performed better in the 1800 sample size. Furthermore, the increase in the sample size affected techniques based on CTT rather than IRT. Also, the findings demonstrated that the techniques’ Type 1 error rates were lower while their statistical power rates were higher under conditions where the test length was 80, and the sample sizes were not equal.

Effects of Education on Differential Item Functioning on the 15-Item Modified Korean Version of the Boston Naming Test.

ObjectiveEducation is expected to have an effect on differential item functioning (DIF) on the 15-item Modified Boston Naming Test in the Korean version of the Consortium to Establish a Registry for Alzheimer's Disease Assessment Packet (BNT-KC). However, no study has examined DIF in the BNT-KC.MethodsWe used the item response theory to investigate the impact of education on the DIF in the BNT-KC among elderly individuals with or without dementia (n=720). A two-parameter item response model was used to determine the difficulty and discrimination parameters of each item. The Benjamini-Hochberg procedure was used to address the risk of Type I errors on multiple testing.ResultsFour items, “mermaid,” “acorn,” “compass,” and “pomegranate” continued to demonstrate DIF after controlling for multiple comparisons. Those with low education levels were more likely to error on “mermaid” and “compass,” while those with high education levels were more likely to error on “acorn” and “pomegranate.” “Hand” and “red pepper” were too easily identified to be used for detecting dementia patients. “Monk's hat” and “pomegranate” were less discriminating than other items, limiting their usefulness in clinical setting.ConclusionThese findings may provide useful information for the development of a revised version of the BNT-KC to help clinicians make diagnostic decisions more accurately.

DSM-IV depressive symptom expression among individuals with a history of hypomania: A comparison to those with or without a history of mania

In an effort to advance an understanding of the phenomenology of bipolar II depression, the current study used methods based in item response theory to evaluate differences in DSM-IV depression symptom endorsement in an epidemiological sample of individuals with a history of hypomania (i.e., bipolar II depression) in comparison to: a) individuals with a history of mania (i.e., bipolar I depression), and b) individuals without a history of hypomania or mania (i.e., unipolar depression). Clinical interview data were drawn from a subsample ( n = 13,753) of individuals with bipolar II, bipolar I, or unipolar depression who had participated in the National Epidemiologic Survey on Alcohol and Related Conditions. A two-parameter item response model was used to estimate differential item functioning (DIF) between these groups. Differences in severity parameter estimates revealed that suicidal ideation/attempt was less likely to be endorsed across most levels of depression severity in bipolar II versus bipolar I disorder. There were no significant differences between groups on the remaining DSM-IV symptoms. Although preliminary, current study data are consistent with recent assertions that depression may be understood as a clinical phenomenon that is consistent across the major affective disorders. An exception to this conclusion may be in the area of suicidal ideation, which requires additional attention.

Differential item functioning of DSM‐IV depressive symptoms in individuals with a history of mania versus those without: an item response theory analysis

Although major depression is characteristic of both bipolar disorder and major depressive disorder, there is disagreement as to whether there are distinct features of depression that differentiate these two conditions. The primary aim of this study was to use methods based in item response theory to evaluate differences in DSM-IV depression symptom endorsement in an epidemiological sample of individuals with a history of mania (i.e., bipolar depression) versus those without (i.e., unipolar depression). Clinical interview data were drawn from a subsample (n = 13,058) of individuals with bipolar or unipolar depression who had participated in the National Epidemiologic Survey on Alcohol and Related Conditions. Using these data, a two-parameter item response model was used to estimate differential item functioning of DSM-IV depressive symptoms between these two groups. Differences in severity parameter estimates revealed that suicidal ideation and psychomotor disturbance were more likely to be endorsed across most levels of depression severity in bipolar versus unipolar depression. Differences in discrimination parameter estimates revealed that fatigue was significantly less discriminating in bipolar versus unipolar depression. Equating for level of depression symptom severity, study results revealed that suicidal ideation and psychomotor disturbance are endorsed more frequently in bipolar versus unipolar depression. Study data also suggested that fatigue may be endorsed more frequently in unipolar relative to bipolar samples at moderate (versus low or high) levels of depression symptom severity.

Impact of Missing Data on Person—Model Fit and Person Trait Estimation

The purpose of this research was to examine the effects of missing data on person—model fit and person trait estimation in tests with dichotomous items. Under the missing-completely-at-random framework, four missing data treatment techniques were investigated including pairwise deletion, coding missing responses as incorrect, hotdeck imputation, and model-based imputation. Person traits were estimated using the two-parameter item response model. Overall, missing data increased the difficulty in assessing person—model fit for both model-fitting and model-misfitting persons. The higher the proportion of missing data, the larger the number of persons incorrectly diagnosed. Among the four techniques, the pairwise deletion method performed best in recovering person—model fit and person trait level. Treating missing responses as incorrect caused the examinees with missing data to not fit the measurement model, thus invalidating the person trait estimates.

Considerations for Measuring Functioning of the Elderly: IRM Dimensionality and Scaling Analysis

Modern measurement methods were applied in this study to examine the properties of a measure of functioning of the elderly. Measures of functioning form an essential element in health services and outcomes research. Several implications for scale development and improved score precision are presented in this case study. This study examined the structure of responses to the Older Americans Resources and Services (OARS) Activities of Daily Living (ADL) and Instrumental Activities of Daily Living (IADL) scales using item response model (IRM) analysis methods. The analysis illustrates the extension of IRM dimensionality and item analysis to health scales in general. Attention is given to the underlying theory and appropriate interpretation of these methods for health measurement. Data were taken from 1364 elderly Canadians participating in the caregiver component of the Canadian Study of Health and Aging (CSHA). The fit of a two-parameter logistic IRM was compared with a one-parameter (Rasch) model for these data. The dimensionality of responses to the scale was evaluated with an approximate χ2 test of residuals after fitting an IRM based on non-linear factor analysis. Results confirm that ADL and IADL item sets differ in the degree of disability they measure and are well represented as separate dimensions using a two-parameter IRM. Implications are drawn concerning the adequacy of the OARS disability measure for health surveys, while more general conclusions cover the precision of IRM based optimal scoring for functional disability measures.

小学校３年生から５年生の算数学力尺度の作成

To construct a scale for arithmetic ability, three forms of test were built and administered to grades three through five. Applying the two-parameter item response model, all parameters of the model were estimated simultaneously for three test forms. Thus, it was possible to compare the difficulty and the discriminating power of all items in a common scale. Accuracy of ability estimation of each test, which was evaluated by the test information function, revealed that precision of estimation was enough for practical use. In the common scale, the average of estimated ability was--.35 for grade three, 0.0 for grade four, and 1.48 for grade five, showing that the difference between grade three and four was small, compared to that between grade four and five. Possible applications of the scale were also discussed.

An Iterative Procedure for Linking Metrics and Assessing Item Bias in Item Response Theory

The presence of biased items may seriously affect methods used to link metrics in item response theory. An iterative procedure designed to minimize this methodological problem was examined in a monte carlo investigation using the two-parameter item re sponse model. The iterative procedure links the scales of independently calibrated parameter estimates using only those items identified as unbiased. Two methods for transforming parameter estimates to a common metric were incorporated into the iterative procedure. The first method links scales by equating the first two moments of the distributions of estimated item diffi culties. The second method determines the linking transformation by minimizing differences across IRT characteristic curve estimates. Results indicate that it erative linking provides a substantial improvement in item bias detection over the noniterative approach. Index terms: Item bias, Item response theory, Iterative method, Linking, Metric linking, Two-parameter item response model.

Effects of Instructional Variables on Language Ability of Preschool Children

The effects of instructional level, teacher’s knowledge of children’s ability, amount of teacher-initiated activity, and age on language ability were investigated. The subjects were 597 Head Start children ranging from 3 years to 6 years, 5 months. Instructional level and language ability were measured on the same scale using the two-parameter Item Response Model. This made it possible to compare directly the child’s ability level and the instructional level. The relationship between the instructional variables and language ability was examined by using multiple regression equations in a structural equation model. The results indicate that instructional level, accuracy of teacher’s knowledge of children’s skills, amount of teacher-initiated activity, and age have a direct effect on children’s language ability.

Effects of Instructional Variables on Language Ability of Preschool Children

The effects of instructional level, teacher’s knowledge of children’s ability, amount of teacher-initiated activity, and age on language ability were investigated. The subjects were 597 Head Start children ranging from 3 years to 6 years, 5 months. Instructional level and language ability were measured on the same scale using the two-parameter Item Response Model. This made it possible to compare directly the child’s ability level and the instructional level. The relationship between the instructional variables and language ability was examined by using multiple regression equations in a structural equation model. The results indicate that instructional level, accuracy of teacher’s knowledge of children’s skills, amount of teacher-initiated activity, and age have a direct effect on children’s language ability.

The identification of people for whom job descriptive index scores are inappropriate

Occasionally, personal and situational factors can adversely affect measurement accuracy of job satisfaction questionnaires. Fatigue, misunderstood instructions, and poor reading comprehension are examples where at least some observed responses to questionnaires may not be meaningfully related to an individual's existing level of job satisfaction. Appropriateness indices ( M. V. Levine & D. Rubin, Journal of Educational Statistics, 1979, 4, 269–290) are a method of quantifying the probability of response patterns in order to identify those questionnaires containing unexpected responses. The indices in this study, based on a two-parameter item response model, showed good hit rates for identifying experimentally distorted response patterns. Further predictions concerning lower scale variance in samples with lower appropriateness indices were also substantiated in three large samples of survey data. These results are discussed in relation to potential improvements in the appropriateness index, the systematic study of undesirable response distortion, and applications of the research.