Item Response Analysis Research Articles

Item response analysis of Alzheimer's Disease Assessment Scale.

Alzheimer's Disease Assessment Scale-cognitive subscale (ADAS-cog) suffers from low sensitivity in detecting changes in Alzheimer's disease progression in clinical trials of disease-modifying treatments. A comprehensive psychometric analysis of the items in ADAS-cog assessment can help in identifying and improving the insensitive items. Item response theory provides a suitable framework for investigating the ADAS-cog items; however, it requires prior information on the underlying latent construct for reliable analysis. In this study, we perform an exploratory item response analysis to investigate the latent construct underlying the relationships between the ADAS-cog item responses. The results indicate that the underlying latent construct of ADAS-cog is multidimensional in nature with the latent factors measuring cognitive declines in several domains (such as memory, praxis, and language domains).

An Analysis of Responses on the Earning Capacity Assessment Form-2

The Earning Capacity Assessment Form-2 assesses the factors that inhibit and facilitate an individual's return to work following an acquired disability (e.g., accident, misadventure). The purpose of this article is to provide the first item response analysis of the form. Data on Australian and US accident victims (N = 110) from two vocational assessment practices were used as a basis for evaluation. An item-response analysis was conducted using the Quest package for the 14 category responses. Results indicated that (a) the items function as inhibitors rather than drivers; and (b) they form a coherent underlying dimension of impairment. Problems in cognition and vocational adjustment issues were major markers for work incapacity.

LittlEARS® early speech production questionnaire validation in hearing Turkish infants

AimThe LEESPQ validated on hearing, German infants is a standardized tool examining preverbal speech development. This study aims to validate the LEESPQ on hearing, Turkish infants.Materials and methodsThis is a prospective, cross-sectional validation study using the LEESPQ in Turkish. The LEESPQ was filled in once for each hearing infant. Data for ≥10 infants was collected at 19 test intervals, ≥0–1, ≥1–2……, ≥18–19 months. Psychometric validation was performed through confirmatory factor analysis, item response analysis, item analysis, and analysis of reliability and validity.ResultsThe LEESPQ was found to be gender independent, have high predictive accuracy and almost exclusively assess speech production ability. A very high correlation between total score and chronological age means score can be interpreted as child's speech production developmental age. Expected and minimum scores were defined for each monthly interval.ConclusionThe LEESPQ (Turkish) has clinical value to confirm typical speech production development and detect potential problems.

The evolution of illness phases in schizophrenia: A non-parametric item response analysis of the Positive and Negative Syndrome Scale

BackgroundThe Positive and Negative Syndrome Scale (PANSS) assesses multiple domains of schizophrenia. Evaluation of each of these domains was conducted to assess differences in the characteristics of psychopathology and their relative predominance in sub-populations. MethodSubjects (N=1,832) with DSM-IV schizophrenia were represented in three sub-populations: First Episodes, n=305, Chronic Inpatients, n=694, and Ambulatory Outpatients, n=833. Nonparametric Item Response Analysis (IRT) was performed with Option Characteristic Curves (OCC), Item Characteristic Curves (ICC), slopes and item biserial correlation. Items were characterized as Very Good, Good, or Weak based on specified operational criteria for item selection. ResultsFirst episode patients were represented by negative, disorganized hostility and anxiety. Some negative domain items (Poor Rapport, Passive/Apathetic Social Withdrawal) and most positive domain items were scored as Weak. For chronic inpatients, all items of the anxiety domain and some items of the positive domain (Suspiciousness/Persecution, Stereotyped Thinking, Somatic Concerns) were Weak; for all other domains, items were Very Good or Good. For ambulatory outpatients, most items in the anxiety and hostility domain were scored as Weak. The majority of PANSS items were either Very Good or Good at assessing the overall illness severity: chronic inpatients (73.33%, 22 items), first episodes (60.00%, 18 items), and only 46.67% (14 items) in the ambulatory group. ConclusionFindings confirm differences in symptom presentation and predominance of particular domains in subpopulations of schizophrenia. Identifying symptom domains characteristic of subpopulations may be more useful in assessing efficacy endpoints than total or subscale scores.

Advancing methodology in the study of HIV status disclosure: The importance of considering disclosure target and intent

Disclosure of HIV status has been the focus of three decades of research, which have revealed its complex relations to many behaviors involved in HIV prevention and treatment, and exposed its central role in managing the HIV epidemic. The causes and consequences of disclosure acts have recently been the subject of several theoretical models. Although it is acknowledged that individual disclosure events are part of a broader process of disclosing one's HIV status to an increasing number of people, this process has received less theoretical attention. In quantitative studies of disclosure, researchers have often implicitly assumed that disclosure is a single unidimensional process appropriately measured via the total number of one's disclosure acts. However, there is also evidence that disclosure may have different causes and consequences depending on the types of actors involved (e.g. family members, friends) and on the presence or absence of the discloser's intention, suggesting that the unidimensionality assumption may not hold. We quantitatively examined the dimensionality of voluntary and involuntary disclosure to different categories of actors, using data collected via structured interviews in the spring of 2010 from 158 people living with HIV in Kilimanjaro, Tanzania. For voluntary disclosure, nonparametric item response analyses identified two multi-category clusters, family and community, and two single-category dimensions, partner and children. Involuntary disclosure consisted of several single- or two-category dimensions. Correlation analyses between the resulting disclosure dimensions and stigma and social support revealed distinct relationships for each disclosure dimension. Our results suggest that treating disclosure as a unidimensional construct is a simplification of disclosure processes that may lead to incorrect conclusions about disclosure correlates. We therefore recommend examining disclosure acts jointly to identify sample-specific dimensions before examining causes and consequences of disclosure. We propose a methodology for investigating disclosure processes, and recommend its adoption in future disclosure studies.

A New Statistical Approach for Quality of Life Questionnaires in the Assessment of Non-Small-Cell Lung Cancer Cuban Patients

Objectives: To evaluate the dimensionality and item characteristics of the European Organization for the Research and Treatment of Cancer Quality of Life Core Questionnaire (EORTC QLQ-C30) and the lung cancer module (QLQ-LC13) and explore the possibility of reduction of the scales. Methods: We analyzed the answers recorded for the QLQ-C30 and QLQ-LC13 in patients diagnosed with non-small-cell lung cancer (NSCLC) participating in 4 Cuban multicenter clinical trials. We assessed the dimensionality underlying both scales with a Mokken nonparametric item response analysis. We used the parametric Samejima’s graded response model to assess the item characteristics; we also conducted a confirmatory factor analysis (CFA) to test the dimensionality of both scales. Taking into account the previous results we compared different reduced scales using the Receiver Operator Curves (ROC Analysis). Results: 873 patients with NSCLC that completed the EORTC QLQ-C30 and 840 patients that completed the QLQ-LC13 were included. Mokken analysis of both scales resulted in 1-dimensional scales. All items showed scalability indices over 0.30. The overall scalability for the QLQ-C30 was 0.43, defining a medium scale according to Mokken’s criteria, while the overall scalability of the QLQ-LC13 was 0.44. Unconstrained Samejima’s graded response models showed appropriate fit, with most items of both scales presenting pertinent difficulty and discrimination parameters. The results of the CFA supported an underlying 1-dimensional latent structure for perceived quality of life (QLQ-C30 comparative fit index [CFI]=0.98; root-mean-square error of approximation [RMSEA]=0.05; QLQ-LC13 CFI=0.99 and RMSEA=0.04). All factor loadings were above 0.30. Conclusions: The QLQ-C30 and the QLQ-LC13 represent in patients with lung cancer a 1-dimensional structure of patient-perceived quality of life. All the reduced scales had similar performance compared with both original scales.

Assessment of a Short Diabetes Knowledge Instrument for Older and Minority Adults

The purpose of the study was to assess the performance of a Short Diabetes Knowledge Instrument (SDKI) in a large multi-ethnic sample of older adults with diabetes and to identify possible modifications to improve its ability to document diabetes knowledge. A sample of 593 African American, American Indian, and white female and male adults 60 years and older, with diabetes diagnosed at least 2 years prior, was recruited from 8 North Carolina counties. All completed an interview that included a 16-item questionnaire to assess diabetes knowledge. A subsample of 46 completed the questionnaire a second time at a subsequent interview. Item-response analysis was used to refine the instrument to well-performing items. The instrument consisting of the remaining items was subjected to analyses to assess validity and test-retest reliability. Three items were removed after item-response analysis. Scores for the resulting instrument were lower among minority and older participants, as well as those with lower educational attainment and income. Scores for test-retest were highly correlated. The SDKI (13-item questionnaire) appears to be a valid and reliable instrument to evaluate knowledge about diabetes. Assessment in a multi-ethnic sample of older adults suggests that this instrument can be used to measure diabetes knowledge in diverse populations. Further evaluation is needed to determine whether or not this instrument can detect changes in knowledge resulting from diabetes education or other interventions.

The Bedford Alzheimer nursing-severity scale to assess dementia severity in advanced dementia: a nonparametric item response analysis and a study of its psychometric characteristics.

The Bedford Alzheimer Nursing-Severity Scale (BANS-S) assesses disease severity in patients with advanced Alzheimer's disease. Since Alzheimer is a progressive disease, studying the hierarchy of the items in the scale can be useful to evaluate the progression of the disease. Data from 164 Alzheimer's patients and 186 patients with other dementia were analyzed using the Mokken Scaling Methodology to determine whether respondents can be ordered in the trait dementia severity, and to study whether an ordering between the items exist. The scalability of the scale was evaluated by the H coefficient. Results showed that the BANS-S is a reliable and medium scale (0.4≤H<0.5) for the Alzheimer group. All items with the exception of the item about mobility could be ordered. When later item was eliminated from the scale, the H coefficient decreased indicating that the scalability of the scale in the original form is more accurate than in the shorter version. For the other dementia group, the BANS-S did not fit any of the Mokken Scaling models because the scale was not unidimensional. In this group, a shorter version of the scale without the sleeping cycle item and the mobility item has better reliability and scalability properties than the original scale.

Development and psychometric testing of the online Adolescent Diabetes Needs Assessment Tool (ADNAT)

To report on the development and psychometric testing of the Adolescent Diabetes Needs Assessment Tool. The UK has the fifth largest paediatric diabetes population in the world, but one of the poorest levels of diabetes control, highlighting the need for intervention development. Mixed methods following recommendations for questionnaire design and validation. A total of 171 young people (12-18 years) participated between 2008- 2011. Methods included item selection using secondary framework analysis, item review, pre-testing, piloting and online transfer. Statistical tests assessed reliability using item-total correlations, interitem consistency and test-retest reliability; and validity using blood glucose (HbA1c) levels and the Self-Management of type 1 Diabetes in Adolescence questionnaire. The Adolescent Diabetes Needs Assessment Tool consists of 117 questions divided between six domains of educational and psychosocial support needs. It combines reflective questioning with needs assessment to raise self-awareness to support adolescent decision-making in relation to diabetes self-care. Thirty-six of the questions provide self-care and psychosocial health assessment scores. Face and content validity of the scoring items were all positively evaluated in terms of appropriateness and readability and tests for validity found significant correlations with Self-Management of type 1 Diabetes in Adolescence and weak correlation with HbA1c , which compared favourably with Self-Management of type 1 Diabetes in Adolescence, the only comparable (USA) tool. Item response analysis validated the use of simple additive scores. The Adolescent Diabetes Needs Assessment Tool combines reflective learning with needs assessment to support patient-centred clinical consultations.

A Gitest-Based Analysis and Evaluation in Language Testing—A Case of a Final Exam

This paper analyses and evaluates an English final exam paper through GitestIII, which was developed and popularized by Guangdong University of Foreign Studies. The analysis includes detailed item response analysis for objectives, obtaining such information as reliability, validity, difficulty, discrimination index, etc. With this case study done, language teachers would be encouraged to adopt the application of modern statistics such as Gitest and SPSS in similar exams to get more useful information about the exams, thus guiding the teaching and testing more efficiently and scientifically.

A new Integrated Negative Symptom structure of the Positive and Negative Syndrome Scale (PANSS) in schizophrenia using item response analysis

BackgroundDebate persists with regard to how best to categorize the syndromal dimension of negative symptoms in schizophrenia. The aim was to first review published Principle Components Analysis (PCA) of the PANSS, and extract items most frequently included in the negative domain, and secondly, to examine the quality of items using Item Response Theory (IRT) to select items that best represent a measurable dimension (or dimensions) of negative symptoms. MethodsFirst, 22 factor analyses and PCA met were included. Second, using a large dataset (n=7187) of participants in clinical trials with chronic schizophrenia, we extracted items loading on one or more PCA. Third, items not loading with a value of ≥0.5, or loading on more than one component with values of ≥0.5 were discarded. Fourth, resulting items were included in a non-parametric IRT and retained based on Option Characteristic Curves (OCCs) and Item Characteristic Curves (ICCs). Results15 items loaded on a negative domain in at least one study, with Emotional Withdrawal loading on all studies. Non-parametric IRT retained nine items as an Integrated Negative Factor: Emotional Withdrawal, Blunted Affect, Passive/Apathetic Social Withdrawal, Poor Rapport, Lack of Spontaneity/Conversation Flow, Active Social Avoidance, Disturbance of Volition, Stereotyped Thinking and Difficulty in Abstract Thinking. ConclusionsThis is the first study to use a psychometric IRT process to arrive at a set of negative symptom items. Future steps will include further examination of these nine items in terms of their stability, sensitivity to change, and correlations with functional and cognitive outcomes.

Validating and Improving the Reliability of the EORTC QLQ-C30 Using a Multidimensional Rasch Model

The reliability and validity of the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-C30) has not been examined while taking into account the correlation between subscales. The reliability of the EORTC QLQ-C30 subscales is modest, thus limiting their utility in both clinical and research settings. The purpose of this study was to validate the factor structure of multiple-item subscales of the EORTC QLQ-C30 and to improve their reliability by means of an item response analysis by using the multidimensional partial credit model. A total of 2295 patients with complete data were used for the analysis. One- and nine-dimensional partial credit models were used to fit the data to validate the construct validity of the multiple-item subscales of the QLQ-C30. The model comparison showed that the nine-dimensional factor structure of multiple-item subscales was satisfactory. The multidimensional partial credit model fit data of the multiple-item subscales of the QLQ-C30 reasonably well. The estimated test reliabilities of each domain obtained from the multidimensional approach were higher than those obtained from the unidimensional approach. The constructs represented by the multiple-item subscales of the QLQ-C30 were validated. The improved reliability of the multiple-item subscales of the QLQ-C30 under the multidimensional approach can facilitate their applications in clinical and research settings.

The Role of Numeracy and Intelligence in Health‐Risk Estimation and Medical Data Interpretation

ABSTRACTDespite ample evidence that numeracy is an important influence on patient understanding and use of health‐related information, there is a dearth of studies examining the concept's relationship to other individual differences measures that may underlie complex judgments in the health domain. In this study, we compared the relative contributions of selected extant numeracy measures and general intelligence and other measures to varied judgment and decision‐making outcomes. Two hundred participants completed numeracy items, subscales of the Wechsler Adult Intelligence Scales, the need for cognition scale, and four relevant outcome measures including risk estimation and medical data interpretation. A numeracy scale constructed using item response and confirmatory factor analyses was consistently the strongest predictor across all outcome measures and accounted for unique variance over and above general intelligence. The results support the concept of numeracy as an independent construct that merits consideration in patient communication. Copyright © 2013 John Wiley &amp; Sons, Ltd.

Item response analysis on an examination in anesthesiology for medical students in Taiwan: A comparison of one- and two-parameter logistic models

BackgroundStudent examinations are an essential component of medical education and item analyses are important to assess test quality. Among miscellaneous psychometric theories used for test analyses, item response theory is more flexible and versatile than other theories. This study aimed to apply item response models to analyze an anesthesiology examination for medical and dental students. MethodsThis examination included 50 items that were administered to 170 5th- and 6th-year medical and dental students. One- and two-parameter logistic (1-PL and 2-PL) item response models were used to conduct item analyses of the examination. Fit statistics were examined to exclude misfit items and evaluate test reliability. Goodness-of-fit analyses were used to select the model having the better fit to data. Examinee's ability and item difficulty were estimated and then expressed on the common scale. Potentially differential items were detected using logistic regression. ResultsThe goodness-of-fit analysis revealed that, in our case, the 1-PL model was more suitable for item response analyses. No misfit item was noted and the test reliability was 0.81 (1-PL model). The mean examinee's ability was set at 0 by definition [standard deviation (SD) = 0.61] and the mean item difficulty was −2.08 (SD = 1.93). There were 24 items with a difficulty level lower than the least able examinee, and three items had a difficulty level higher than the most able examinee. Four potentially differential items were identified. ConclusionItem response models are useful for medical test analyses and provide valuable information about model comparisons and identification of differential items other than test reliability, item difficulty, and examinee's ability.

Validation of a 10-item Care-related Regret Intensity Scale (RIS-10) for Health Care Professionals

Regret after one of the many decisions and interventions that health care professionals make every day can have an impact on their own health and quality of life, and on their patient care practices. To validate a new care-related regret intensity scale (RIS) for health care professionals. Retrospective cross-sectional cohort study with a 1-month follow-up (test-retest) in a French-speaking University Hospital. A total of 469 nurses and physicians responded to the survey, and 175 answered the retest. RIS, self-report questions on the context of the regret-inducing event, its consequences for the patient, involvement of the health care professionals, and changes in patient care practices after the event. We measured the impact of regret intensity on health care professionals with the satisfaction with life scale, the SF-36 first question (self-reported health), and a question on self-esteem. On the basis of factor analysis and item response analysis, the initial 19-item scale was shortened to 10 items. The resulting scale (RIS-10) was unidimensional and had high internal consistency (α=0.87) and acceptable test-retest reliability (0.70). Higher regret intensity was associated with (a) more consequences for the patient; (b) lower life satisfaction and poorer self-reported health in health care professionals; and (c) changes in patient care practices. Nurses reported analyzing the event and apologizing, whereas physicians reported talking preferentially to colleagues, rather than to their supervisor, about changing practices. The RIS is a valid and reliable measure of care-related regret intensity for hospital-based physicians and nurses.

Psychometrical assessment and item analysis of the General Health Questionnaire in victims of terrorism.

There is a need to assess the psychiatric morbidity that appears as a consequence of terrorist attacks. The General Health Questionnaire (GHQ) has been used to this end, but its psychometric properties have never been evaluated in a population affected by terrorism. A sample of 891 participants included 162 direct victims of terrorist attacks and 729 relatives of the victims. All participants were evaluated using the 28-item version of the GHQ (GHQ-28). We examined the reliability and external validity of scores on the scale using Cronbach's alpha and Pearson correlation with the State-Trait Anxiety Inventory (STAI), respectively. The factor structure of the scale was analyzed with varimax rotation. Samejima's (1969) graded response model was used to explore the item properties. The GHQ-28 scores showed good reliability and item-scale correlations. The factor analysis identified 3 factors: anxious-somatic symptoms, social dysfunction, and depression symptoms. All factors showed good correlation with the STAI. Before rotation, the first, second, and third factor explained 44.0%, 6.4%, and 5.0% of the variance, respectively. Varimax rotation redistributed the percentages of variance accounted for to 28.4%, 13.8%, and 13.2%, respectively. Items with the highest loadings in the first factor measured anxiety symptoms, whereas items with the highest loadings in the third factor measured suicide ideation. Samejima's model found that high scores in suicide-related items were associated with severe depression. The factor structure of the GHQ-28 found in this study underscores the preeminence of anxiety symptoms among victims of terrorism and their relatives. Item response analysis identified the most difficult and significant items for each factor.

Performance of the 10-item Center for Epidemiologic Studies Depression scale for caregiving research.

Objectives:The Center for Epidemiologic Studies Depression (CESD) scale has been useful in a broad spectrum of health research on patient and population outcomes. A brief version is used when depressive symptoms are not the primary focus. Rasch (item response) analysis previously demonstrated potential problems with positively worded items. We tested the 10-item CESD (CESD-10) scale and considered an 8-item version with both psychometric and Rasch analyses.Methods:This was a special sample of 2067 caregivers from three existing US databases. We describe item response patterns and internal constancy in addition to Rasch scale results.Results:There were few problems with missing data, and internal consistency was high (alpha = 0.86–0.88) for both CESD versions. Rasch analysis indicated that one of the positive items (“hopeful about future”) could be dropped.Conclusions:We partly confirmed prior work that suggested dropping positive items for the CESD-10. Among caregivers, item-level problems and scaling problems seem minimal. At present, there is not a strong rationale for dropping the CESD-10 positive items: the one poorly performing positive item might be explained by the special caregiver sample.

Psychometric Evaluation of an Inpatient Consumer Survey Measuring Satisfaction with Psychiatric Care

Measurement of consumers' satisfaction in psychiatric settings is important because it has been correlated with improved clinical outcomes and administrative measures of high-quality care. These consumer satisfaction measurements are actively used as performance measures required by the accreditation process and for quality improvement activities. Our objectives were (i) to re-evaluate, through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), the structure of an instrument intended to measure consumers' satisfaction with care in psychiatric settings and (ii) to examine and publish the psychometric characteristics, validity and reliability, of the Inpatient Consumer Survey (ICS). To psychometrically test the structure of the ICS, 34 878 survey results, submitted by 90 psychiatric hospitals in 2008, were extracted from the Behavioral Healthcare Performance Measurement System (BHPMS). Basic descriptive item-response and correlation analyses were performed for total surveys. Two datasets were randomly created for analysis. A random sample of 8229 survey results was used for EFA. Another random sample of 8261 consumer survey results was used for CFA. This same sample was used to perform validity and reliability analyses. The item-response analysis showed that the mean range for a disagree/agree five-point scale was 3.10-3.94. Correlation analysis showed a strong relationship between items. Six domains (dignity, rights, environment, empowerment, participation, and outcome) with internal reliabilities between good to moderate (0.87-0.73) were shown to be related to overall care satisfaction. Overall reliability for the instrument was excellent (0.94). Results from CFA provided support for the domains structure of the ICS proposed through EFA. The overall findings from this study provide evidence that the ICS is a reliable measure of consumer satisfaction in psychiatric inpatient settings. The analysis has shown the ICS to provide valid and reliable results and to focus on the specific concerns of consumers of psychiatric inpatient care. Scores by item indicate that opportunity for improvement exists across healthcare organizations.

L07 The functional rating taskforce for pre-huntington's disease: development of the furst-21 scale

BackgroundThere is a need for clinical scales specifically designed to measure the earliest clinical manifestations of Huntington disease (HD) and to track early changes in clinical symptoms in HD gene...

Assessing DSM-IV symptoms of panic attack in the general population: An item response analysis

Unexpected panic attacks may represent a non-specific risk factor for future depression and anxiety disorders. The examination of panic symptoms and associated latent severity levels may lead to improvements in the identification, prevention, and treatment of panic attacks and subsequent psychopathology for 'at risk' individuals in the general population. The current study utilised item response theory to assess the DSM-IV symptoms of panic in relation to the latent severity level of the panic attack construct in a sample of 5913 respondents from the National Epidemiologic Survey on Alcohol and Related conditions. Additionally, differential item functioning (DIF) was assessed to determine if each symptom of panic targets the same level of latent severity between different sociodemographic groups (male/female, young/old). Symptoms indexing 'choking', 'fear of dying', and 'tingling/numbness' are some of the more severe symptoms of panic whilst 'heart racing', 'short of breath', 'tremble/shake', 'dizzy/faint', and 'perspire' are some of the least severe symptoms. Significant levels of DIF were detected in the 'perspire' symptom between males and females and the 'fear of dying' symptom between young and old respondents. The current study was limited to examining cross-sectional data from respondents who had experienced at least one panic attack across their lifetime. The findings of the current study provide additional information regarding panic symptoms in the general population that may enable researchers and clinicians to further refine the detection of 'at-risk' individuals who experience threshold and sub-threshold levels of panic.