Classical Test Theory Analysis Research Articles

Determining the scoring validity of a co-constructed CEFR-based rating scale

Considering scoring validity as encompassing both reliable rating scale use and valid descriptor interpretation, this study reports on the validation of a CEFR-based scale that was co-constructed and used by novice raters. The research questions this paper wishes to answer are (a) whether it is possible to construct a CEFR-based rating scale with novice raters that yields reliable ratings and (b) allows for a uniform interpretation of the descriptors. Additionally, this study focuses on the question whether co-constructing a rating scale with novice raters helps to stimulate a shared interpretation of the descriptors over time. For this study, six novice raters employed a CEFR-based scale that had been co-constructed by themselves and 14 peers to rate 200 spoken and written performances in a missing data design. The quantitative data were analysed using item response theory, classical test theory and principal component analysis. The focus group data, collected after the rating process, were transcribed and coded using both a priori and inductive coding. The results indicate that novice raters can reliably use the CEFR-based rating scale, but that the interpretations of the descriptors, in spite of training and co-construction, are not as homogeneous as the inter-rater reliability would suggest.

Exploring the construct validity of the Patient Perception Measure - Osteopathy (PPM-O) using classical test theory and Rasch analysis.

BackgroundEvaluation of patients’ experience of their osteopathic treatment has recently been investigated leading to the development of the Patient Perception Measure – Osteopathy (PPM-O). The aim of the study was to investigate the construct validity of the PPM-O.MethodsPatients presenting to osteopathy student-led teaching clinics at two Australian universities were asked to complete two questionnaires after their treatment: a demographic questionnaire and the PPM-O. Confirmatory factor analysis (CFA) and Rasch analysis were used to investigate the construct validity of the PPM-O.ResultsData from the present study did not fit the a-priori 6-domain structure in the CFA. Modifications to the 6-domain model were then made based on the CFA results, and this analysis identified two factors: 1) Education & Information (9 items); and 2) Cognition & Fatigue (6 items). These two factors were Rasch analysed individually. Two items were removed from the Cognition & Fatigue factor during the analysis. The two factors independently were unidimensional.ConclusionsThe study produced a 2-factor, 13-item questionnaire that assesses the patients’ perception of their osteopathic treatment using the items from a previous questionnaire. The results of the current study provide evidence for the construct validity of the PPM-O and the small number of items makes it feasible to implement into both clinical and research settings. Further research is now required to establish the measures’ validity in a variety of patient populations.Electronic supplementary materialThe online version of this article (doi:10.1186/s12998-015-0055-x) contains supplementary material, which is available to authorized users.

The Development And Validation Of The Test Of Astronomy STandards (TOAST)

<p>The Test Of Astronomy STandards (TOAST) is a comprehensive assessment instrument designed to measure students general astronomy content knowledge. Built upon the research embedded within a generation of astronomy assessments designed to measure single concepts, the TOAST is appropriate to measure across an entire astronomy course. The TOASTs scientific content represents a consensus of expert opinion about what students should know from three different groups: the American Association for the Advancement of Science, the National Research Council, and the American Astronomical Society. The TOASTs reliability and validity are established by results from Cronbach alpha and classical test theory analyses, a review for construct validity, testing for sensitivity to instruction, and numerous rounds of expert review. As such the TOAST can be considered a valuable tool for classroom instructors and discipline based education researchers in astronomy across a variety of learning environments.</p>

PCQoL: A Quality of Life Assessment Measure for Pachyonychia Congenita.

Pachyonychia congenita (PC) is a rare but often debilitating, dominantly inherited disorder. New treatments require more accurate instruments for evaluating changes in the quality of life in these patients. This study was undertaken to develop and validate a quality of life questionnaire for PC patients (PCQoL). Relevant factors influencing quality of life in PC patients were identified and incorporated into the well-recognized, general questionnaire, the Dermatology Life Quality Index (DLQI), to establish a disease-specific measure, the PCQoL. Classical test theory (CTT) and Rasch analysis (RA) were used to analyze and validate the PCQoL. CTT analysis established test-retest reliability and internal consistency for the PCQoL. Concurrent and construct validity for the DLQI and the PCQoL were also validated. Chi-square-based infit and outfit statistics indicated that the Rasch model fits the observed responses very well. RA reconfirmed reliability, internal consistency, reasonable homogeneity, construct validity, and the presence of three RA-based domains. The PCQoL questionnaire is a measure validated by both CTT and RA. It appears to be a valuable tool in measuring quality of life modifications in PC individuals with keratoderma.

Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory

AbstractGeneralizability theory (G theory) provides a broad conceptual framework for social sciences such as psychology and education, and a comprehensive construct for numerous measurement events by using analysis of variance, a strong statistical method. G theory, as an extension of both classical test theory and analysis of variance, is a model which can deal with multiple sources of error. In conducting the analysis of the G theory, there are several software programs that can be used such as GENOVA, SPSS, SAS, EduG, and G-String. In this study, the general perspectives of G theory are first explained broadly. Then, the SPSS and EduG software programs are used to conduct generalizability analyses on the data obtained from the answers of 30 students (p) to nine open-ended questions (i) as rated by three raters (r). There are three different designs in the study. Two of them are random effects designs, pxixr and pxi:r, and the last one is pxixr design using a fixed rater . According to the findings from the study, SPSS and EduG give the same results for variance component estimates as well as for G (Generalizability) and D (Decision) studies of all designs, as expected. Besides comparing the program outputs, their weaknesses and strengths were also discussed regarding different designs and data sets in this study.Keywords: Generalizability Theory * G Study * D Study * SPSS * EduGG theory has formed a comprehensive structure by employing variance analysis which provides a broad conceptual framework for social sciences such as psychology and education (Brennan, 2000, 2001a; Cronbach, Gleser, Nanda, & Rajaratnam, 1972; Shavelson & Webb, 1991). It is also a powerful statistical tool for situations where there are numerous measurements. The theory, as an extension of classical test theory and variance analysis, stands as a model where multiple sources of error can be handled (Brennan, 2001a; Shavelson & Webb, 1991).Generalizability (G) TheoryThe reliability of measurement results in education and psychology was previously examined using classical test theory (CTT) in general. It is assumed in CTT that the observed score is composed of the actual score with no separable score for error. The restriction of this assumption, especially in performance measurements where the probability of the existence of more than one source of error is high, reveals the importance of G theory in which more than one source of error is handled and can be predicted simultaneously (Brennan, 2000). Another advantage of G theory in using performance assessment is that while there is a restrictive parallel assumption in CTT, randomly parallel assumption is adopted in G theory (Brennan, 2011; Kretchmar, 2006). The main aim of G theory is to generalize the scores of a specific measurement tool from a specific group to the universe of generalization which consist of 1) the universe of admissible observations and generalizability studies (G studies), 2) the universe of G studies and decision studies (D studies). While G studies provide an estimate of the generalizability coefficient of variances from all facets and this coefficient includes the examinee's universe score, D studies enable one to examine the interactions among all applicable facets (tasks, raters, observations, etc.) and the subject of measurement for calculating the dependability coefficient (Brennan, 2000; Crocker & Algina, 1986; Hsu, 2012).G theory has four main advantages compared to CTT. 1) It provides simultaneous evaluation of test-retest reliability, internal consistency, inter-rater reliability, and convergent validity. 2) It enables estimates of both individual measurement facets and interaction effects. 3) When assessing an examinee's performance, it gives information about the quality of their absolute structural level of knowledge as well as ranking this information in order. 4) It allows researchers to optimize the reliability of an assessment within the cost constraints of time and money. …

Classical Test Theory and Rasch Analysis Validation of the Upper Limb Functional Index in Subjects With Upper Limb Musculoskeletal Disorders

ObjectiveTo perform a comprehensive analysis of the psychometric properties and dimensionality of the Upper Limb Functional Index (ULFI) using both classical test theory and Rasch analysis (RA). DesignProspective, single-group observational design. SettingFreestanding rehabilitation center. ParticipantsConvenience sample of Italian-speaking subjects with upper limb musculoskeletal disorders (N=174). InterventionsNot applicable. Main Outcome MeasuresThe Italian version of the ULFI. Data were analyzed using parallel analysis, exploratory factor analysis, and RA for evaluating dimensionality, functioning of rating scale categories, item fit, hierarchy of item difficulties, and reliability indices. ResultsParallel analysis revealed 2 factors explaining 32.5% and 10.7% of the response variance. RA confirmed the failure of the unidimensionality assumption, and 6 items out of the 25 misfitted the Rasch model. When the analysis was rerun excluding the misfitting items, the scale showed acceptable fit values, loading meaningfully to a single factor. Item separation reliability and person separation reliability were .98 and .89, respectively. Cronbach alpha was .92. ConclusionsRA revealed weakness of the scale concerning dimensionality and internal construct validity. However, a set of 19 ULFI items defined through the statistical process demonstrated a unidimensional structure, good psychometric properties, and clinical meaningfulness. These findings represent a useful starting point for further analyses of the tool (based on modern psychometric approaches and confirmatory factor analysis) in larger samples, including different patient populations and nationalities.

Measuring social support in patients with advanced medical illnesses: An analysis of the Duke-UNC Functional Social Support Questionnaire.

To date, no measure of social support has been developed specifically for either palliative care or oncology settings. The present study examined the psychometric properties of the Duke-University of North Carolina Functional Social Support Questionnaire (DUFSS) in order to (1) assess the adequacy of the scale in the context of severe medical illness and (2) evaluate whether a brief subset of items might generate roughly comparable utility. The 14-item DUFSS was administered to 1,362 individuals with advanced cancer or AIDS. Classical test theory (CTT) and item response theory (IRT) analyses were utilized to develop an abbreviated version of the DUFSS that maintained adequate reliability and validity and might increase the feasibility of its administration in a palliative care setting. The reliability and concurrent validity of the DUFSS-5 were evaluated in a separate validation sample of patients with advanced cancer. Analyses generated a five-item version of the DUFSS (the DUFSS-5) that collapsed response levels into only three options, instead of five. Correlations between the DUFSS-5 and measures of depression, quality of life, and desire for hastened death, as well as regression models testing the main-effect and buffering models of social support, provided support for the utility of the DUFSS-5. Both the DUFSS and the abbreviated DUFSS-5 appear to have adequate reliability and validity in this setting. Moreover, the DUFSS-5 represents a potentially important option for healthcare researchers, particularly for those working in palliative care settings where issues of patient burden are paramount. Such analyses are critical for advancing the development and refinement of psychosocial measures, but have often been neglected.

The Edinburgh Postnatal Depression Scale as a measure for antenatal dysphoria

Background: Data arising from the Edinburgh Postnatal Depression Scale (EPDS), used as screening tool for perinatal depression, have been analysed as unidimensional in some studies and multidimensional in others. This study evaluates the dimensionality and item properties of the EPDS. Methods: 920 women recruited in antenatal clinics from two government tertiary hospitals completed the English EPDS between 26 and 28 weeks of pregnancy. Classical test theory and factor analyses were used to evaluate dimensionality. Item response theory was used to investigate item functioning. Results: The EPDS was essentially unidimensional. It has a common factor, ‘antenatal dysphoria’ and 3 subdomains ‘depressive’ (items 7–10), ‘anxiety’ (items 3–5) and ‘anhedonic symptoms’ (items 1, 2). Item 8 (I have felt sad or miserable) (discrimination = 3.13. SE = 0.24), item 9 (I have been so unhappy that I have been crying) (discrimination = 2.39. SE = 0.17) and item 10 (The thought of harming myself has occurred to me) (discrimination = 2.27. SE = 0.21) best discriminated participants that had dysphoria. Item 1 (I have been able to laugh and see the funny side of things) (threshold 1 = 1.49, SE = 0.14), item 2 (I have looked forward with enjoyment to things) (threshold 1 = 1.21, SE = 0.11) and item 10 (The thought of harming myself has occurred to me) (threshold 1 = 1.19, SE = 0.21) indicated dysphoria severity. Conclusions: The EPDS measures antenatal dysphoria rather than just depression. It could be used to screen for antenatal depressive, anxiety and anhedonic symptoms. The items discriminate and grade dysphoria unequally. The findings may impact on scale use and interpretation.

Item-Based Psychometrics of the Preschool Behavioral and Emotional Rating Scale

The Preschool Behavioral and Emotional Rating Scale (PreBERS) is an assessment of emotional and behavioral strengths in preschoolers with well-established reliability and validity for educational and clinical application in children with and without disabilities. The present study provides further evidence of psychometric rigor for items and subscales within the PreBERS, including internal consistency, reliability, item quality, and factorial validity. A national sample of 1,292 preschool children whose teachers scored the PreBERS were analyzed using classical test theory, Rasch modeling, and confirmatory factor analysis. Results confirmed that individual items were internally consistent within a four-factor model, and showed consistent item difficulty, discrimination, and fit relative to their respective subscale scores. This analysis provides further confidence for the use of the PreBERS by teachers to determine emotional and behavioral strengths in preschool children.

The Multiple Sclerosis-Fatigue Self– Efficacy (MS-FSE) scale: initial validation

Objective: To examine the validity and sensitivity to change of the Multiple Sclerosis-Fatigue Self-Efficacy scale. Design: A validation study nested within a randomized controlled trial. Setting: Community setting. Participants: Adults with a clinically definite diagnosis of multiple sclerosis and significant fatigue taking part in a randomized controlled trial evaluating a group-based fatigue management programme (FACETS) for people with multiple sclerosis (N=164). Main measures: The 9-item Multiple Sclerosis-Fatigue Self-Efficacy scale was completed at baseline, 1-, 4- and 12 months post intervention. Validity, internal consistency and sensitivity to change were examined using classical test theory and Rasch analysis. Results: Item 3 was unanswered by 6% of respondents as they did not know any other people with multiple sclerosis; remaining analyses were carried out with this item deleted. All response choices were utilised, no floor or ceiling effects were evident and there were few missing responses. Cronbach’s alphas were high (baseline, 0.89; follow-up 1, 0.93; follow-up 2, 0.94; follow-up 3, 0.90). The Multiple Sclerosis-Fatigue Self-Efficacy scale (8-item) demonstrated good sensitivity to change following attendance of the FACETS programme (within participant effect sizes 0.66 and 0.69 and 0.54 at 1, 4, and 12 months follow-up). Principal Components Analysis yielded one component. In the Rasch analysis two items with disordered thresholds were rescored. Item 8 displayed differential item functioning by disability and was combined into a testlet with item 4, resulting in a unidimensional scale. The sample was well targeted to the scale. Conclusion: At a scale level the Multiple Sclerosis-Fatigue Self-Efficacy scale is internally valid and has good sensitivity to change.

Combination of classical test theory (CTT) and item response theory (IRT) analysis to study the psychometric properties of the French version of the Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF).

The study aimed to examine the construct validity and reliability of the Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF) according to both classical test and item response theories. The psychometric properties of the French version of this instrument were investigated in a cross-sectional, multicenter study. A total of 124 outpatients with a substance dependence diagnosis participated in the study. Psychometric evaluation included descriptive analysis, internal consistency, test-retest reliability, and validity. The dimensionality of the instrument was explored using a combination of the classical test, confirmatory factor analysis (CFA), and an item response theory analysis, the Person Separation Index (PSI), in a complementary manner. The results of the Q-LES-Q-SF revealed that the questionnaire was easy to administer and the acceptability was good. The internal consistency and the test-retest reliability were 0.9 and 0.88, respectively. All items were significantly correlated with the total score and the SF-12 used in the study. The CFA with one factor model was good, and for the unidimensional construct, the PSI was found to be 0.902. The French version of the Q-LES-Q-SF yielded valid and reliable clinical assessments of the quality of life for future research and clinical practice involving French substance abusers. In response to recent questioning regarding the unidimensionality or bidimensionality of the instrument and according to the underlying theoretical unidimensional construct used for its development, this study suggests the Q-LES-Q-SF as a one-dimension questionnaire in French QoL studies.

Childhood depression subscales using repeated sessions on Children's Depression Rating Scale - revised (CDRS-R) scores.

Although acute treatments have been shown to be effective in treating early-onset depression, only one-third or thereabouts reach a remission within 3 months. Unfortunately, delayed time to remission in early-onset depression leads to poorer therapeutic outcomes. Clearly, there is a need to identify, diagnose, and provide effective treatment of a depressed patient quickly. A sophisticated understanding of depression subscales and their change over time with treatment could enhance pathways to individualized treatment approaches for childhood depression. Previous studies have found that the clinician-measured instrument, Children's Depression Rating Scale-Revised (CDRS-R) measures multiple subscales (or components) of depression. The aim of this study was to see how these subscales may change over the course of a 12-week study. This knowledge will help determine if dimensions/subscales of childhood depression (paralleling the adult literature) using the subscales derived from factor analysis procedure is useful. We examined two clinical trials in which youth (n=234) with major depressive disorder (MDD) were treated openly with fluoxetine for eight sessions spread over 12 weeks. The CDRS-R was completed based on clinician interviews with parent and child at each session. Classical test theory and component analysis with associated parallel analysis (oblique rotation) were conducted on each week's scores. Although more factors were needed for the baseline and first two therapy sessions, a two-factor solution sufficed thereafter. Depressed facial affect, listless speech, and hypoactivity best defined Factor I, whereas sleep problems, appetite disturbance, physical symptoms, irritability, guilt, and weeping best defined Factor II. All other symptoms cross-loaded almost equally on the two factors. The scale's reliability (internal consistency) improved from baseline to exit sessions (α=0.65-0.91). As a result, the clinicians' assessments of the various symptoms became more highly related to one another. This caused the first eigenvalue to increase from 3.24 to 7.38 and the variance explained to increase (%) from 19% to 43% over sessions. These two factors may reflect 1) clinician-observed signs and 2) reported symptoms of depression. Factor analysis of CDRS-R data in a single session consistently generates a complex and difficult to interpret structure of at least three factors. This makes it very difficult to understand what these factors measure. However, when gathered over additional sessions, the CDRS-R structure tends to simplify to two factors. The reasons for this simplification are as yet unclear and in need of further study.

The association between nonadherence and glycated hemoglobin among type 2 diabetes patients using basal insulin analogs.

BackgroundThe main objective of this study was to investigate the relationship between adherence and both clinical (ie, glycated hemoglobin [HbA1c]) and nonclinical (ie, health status, work impairment, and health care-resource use) health outcomes among type 2 diabetes (T2D) patients using basal insulin.Materials and methodsThe 2012 US National Health and Wellness Survey dataset was used for this study (n=71,141). A total of 1,198 respondents who reported a diagnosis of T2D, were currently using basal insulin, and reported both their HbA1c and level of nonadherence were included in the analyses. Classical test theory and item response theory (IRT) analyses were used to provide evidence for the Morisky Medication Adherence Scale (MMAS) in this population. Adherence was then used as a predictor of HbA1c and nonclinical outcomes using regression modeling, controlling for demographics and health history.ResultsA total of 61.44% of respondents were male, and the mean age was 60.65 (standard deviation 10.74) years. Internal consistency of the eight-item MMAS (MMAS-8) was adequate (Cronbach’s α =0.68), and one factor was retained (eigenvalue =1.80). IRT analyses suggested that the MMAS-8 was most precise for those with high levels of nonadherence. A significant relationship between variables emerged, whereby each point increase in the level of nonadherence was associated with a 0.21 increase in HbA1c (B=0.212, P<0.05). A modest quadratic trend was also observed (B=0.026, P<0.05), indicating that the benefit to HbA1c may taper off at high adherence. Each point of nonadherence was associated with a 4.6%, 20.4%, and 20.9% increase in the number of physician visits, emergency room visits, and hospitalizations, respectively.DiscussionThis study provides evidence that adherence rates are high among patients with T2D using basal insulin, and the MMAS-8 is a reliable and valid tool to assess adherence. Further, the results suggest that HbA1c increases concomitantly with nonadherence, as do poorer health status and health care-resource use.

Rasch analysis of the Iowa Level of Assistance Scale in patients with total hip and knee arthroplasty

The aim of this study was to carry out a psychometric analysis, using both Classical Test Theory and Rasch analysis (RA) methods, of the Iowa Level of Assistance Scale (ILAS) administered in patients with recent total hip arthroplasty (THA) or total knee arthroplasty, to examine its metric properties and provide insights for a refined version. A total of 203 patients who had undergone THA or total knee arthroplasty were assessed using the ILAS for assistance needed during functional activities (ILAS-funct) and need for assistive devices (ILAS-dev) before discharge from the orthopedic ward. The responses were psychometrically analyzed using the Classical Test Theory and RA. Cronbach's α was adequate only for group comparisons (ILAS-funct, 0.82; ILAS-dev, 0.79). The two domains (ILAS-funct and ILAS-dev) showed a good correlation. According to RA rating scale diagnostics, ILAS-funct showed two disordered response category thresholds: of the seven different response levels of 'assistance', only five were appreciably discernible. All five ILAS-funct items fitted the model and did not show either local dependence or differential item functioning across age groups or sex. ILAS-dev presented two unused response categories, which precluded Rasch calibration and subsequent analyses. ILAS-funct showed sound psychometric properties, but the rating system of ILAS-funct could be simplified, at least collapsing the response categories 5 (failed) and 6 (not tested). In ILAS-dev, the presence of unused response categories calls for a reconsideration of its scaling options and methods.

The level of agreement between child self-reports and parent proxy-reports of health-related quality of life in boys with Duchenne muscular dystrophy

To investigate the level of agreement between child self-reports and parent proxy-reports of the health-related quality of life (HRQoL) in boys with Duchenne muscular dystrophy (DMD) using both classical test theory (CTT) and Rasch analysis. A total of 63 boys with DMD and their parents completed the pediatric quality of life inventory version 4.0 child self-report and parent proxy-report of HRQoL, respectively. The data were analyzed using both the CTT (scale-score level) and Rasch analysis (item-level). The intraclass correlation coefficient (ICC, scale-score level) between children and parents showed good to moderate agreement, although parents consistently underestimated their child HRQoL. In Rasch analysis (item-level), 1 out of 8 items was significantly different between children and parents in the physical health scale. Also, 3 out of 15 items were significantly different between those two groups in the psychosocial health scale. By applying both scale-score and item-level analyses, our study seeks to broaden the understanding of the discrepancy of the ratings between child self-reports and parent proxy-reports. The findings could provide further information about the decision-making process when selecting therapy and care programs.

Psychometric properties of the Mastery Scale-Chinese version: Applying classical test theory and Rasch analysis

Aims: This study examines the reliability and validity of the Mastery Scale-Chinese version (MS-C) when applied to three groups diagnosed with major depression, schizophrenia, or HIV/AIDS. Methods: The individuals participating in the study were recruited from outpatient units of a medical center and a municipal hospital in northern Taiwan. The study sample (n = 2009) included 237 patients with depressive disorders, 160 with schizophrenia, and 1612 with HIV/AIDS. The reliability and construct validity of the MS-C was evaluated by confirmatory factor analysis (CFA) and Rasch analysis. Results: The results of the CFA showed that the MS-C has adequate construct validity with all indices meeting the criteria, except for chi-square values. The Rasch analysis supported the four-point rating scale structure and a unidimensional construct of the MS-C. The DIF analysis showed that all items demonstrated stable measurement properties among the different diagnosis groups (major depression, schizophrenia, HIV/AIDS). Conclusion: This study found that MS-C has acceptable psychometric qualities in terms of reliability, construct validity, rating scale performance, and item characteristics when applied to patients with depression, schizophrenia, and HIV/AIDS in Taiwan.

Measurement Properties of the Motivation for Youth Treatment Scale with a Residential Group Home Population

A client’s motivation to receive services is significantly related to seeking services, remaining in services, and improved outcomes. The Motivation for Youth Treatment Scale (MYTS) is one of the few brief measures used to assess motivation for mental health treatment. To investigate if the psychometric properties of the MYTS previously found for youth receiving outpatient treatment would be replicated in a sample of youth entering residential care. The study was conducted at a large residential group care agency in the Midwest with 145 youth who completed the 8-item MYTS upon intake into the group homes. To investigate the psychometrics of the MYTS, we used classical test theory, Rasch modeling, and confirmatory factor analysis approaches. Most of the psychometric properties were strong and replicated previous research in outpatient settings. The primary divergent finding was that the two subscales, problem recognition and treatment readiness were not significantly correlated in the residential sample, in contrast to the moderate correlations found in outpatient samples. The MYTS can be recommended for use in residential settings, however, researchers and practitioners should carefully examine the sub-scale scores. Future research needs to replicate these findings in residential settings and further explore why the two subscales are minimally correlated.

A Rasch analysis of the Hospital Anxiety and Depression Scale (HADS) among cancer survivors.

The Hospital Anxiety and Depression Scale (HADS) is commonly used to assess distress among individuals with cancer. However, previous studies cast doubt on the most appropriate dimensional structure for the HADS, suggesting that 1- or 3-dimensional structures might offer superior fit to the original 2-dimensional one. This article is the first to use Rasch analysis to examine the psychometric properties of the subscales corresponding to each of these alternative structures. The HADS was completed by 1,360 cancer survivors. Rasch analyses were conducted to examine summary and individual model fit statistics, person separation index, response format, item bias, redundancy, and dimensionality. The HADS-Total scale was found to be multidimensional, and it was necessary to remove almost half of the items to achieve fit. Analyses only partially supported the original structure, as both HADS-Anxiety and HADS-Depression showed initial model misfit and item deletion was necessary to achieve fit. Within the 3-dimensional structures, the Rasch statistics for the anxiety subscales were within acceptable range and no adjustment was needed. Analyses did not support adding Item 7 to HADS-Depression. Results supported modified versions of the HADS-Anxiety and HADS-Depression; however, combining all items to form HADS-Total is not recommended. Numerous studies using classical test theory and Rasch analyses have corroborated the exclusion of some items (e.g., Item 7) and appropriateness of the subscales defined by a 3-dimensional structure. Further research is required to identify the incremental validity of potential revised subscales.

SAT0387 Psychometric Characteristics of the Italian Version of the Revised Fibromyalgia Impact Questionnaire Using Classical Test Theory and Rasch Analysis

ObjectivesThe aim of this study was to perform a psychometric analysis of the Italian Fibromyalgia Impact Questionnaire Revised version (FIQR), using both classical test theory (CTT) and Rasch analysis (RA)...

Abstract 1367: Item response theory analysis of the patient satisfaction with cancer-related care: psychometric validation in a multicultural sample of 1,296 participants.

Abstract BACKGROUND: Patient satisfaction (PS) is an important outcome measure of quality of cancer-related care. PS was one of the four core study outcomes of the National Cancer Institute and American Cancer Society funded $25 million multicenter Patient Navigation Research Program (PNRP) to reduce disparities in cancer care. A Patient Satisfaction with Cancer Care (PSCC) measure was developed and validated for the PNRP using classical test theory and principal components analysis (PCA). OBJECTIVE: To calibrate items of the PSCC to facilitate the development of a computerized adaptive testing (CAT) system, which can be used to tailor the PSCC to patients’ satisfaction level based on properties of the items. METHODS: The PCA revealed a unidimensional PSCC measure. Thus, we applied unidimensional item response theory (IRT) models to the 18-item PSCC data from 1,296 participants (73% female; age 18 to 86 years). We fitted two IRT models to the data: An unconstrained graded response model (GRM) and a constrained GRM (i.e., Rasch Model) where all discrimination parameters across items were fixed to be equal. We obtained model fit indices (log-likelihood, AIC &amp; BIC) and performed model comparison through likelihood ratio (LR) test between the unconstrained GRM and the Rasch model. We obtained item and latent trait (i.e., patient satisfaction) parameter estimates, category characteristic curves, operating characteristic curves, and test information curves for the better fitting model. RESULTS: The unconstrained GRM fitted the data significantly better (LR = 828, df = 17, p &amp;lt; 0.001). Item parameter estimates showed strong items discriminating power (α = 0.94 to 2.18). Standard errors (SE) of the item parameter estimates were also small (i.e., mostly around 0.1 for the threshold parameters, and between 0.1 to 0.2 for the discrimination parameters), confirming the precision of the item parameter estimates obtained. CONCLUSIONS: The PSCC is suitable to be delivered through a CAT system where patients will receive tailored optimally selected items to measure their satisfaction levels, and scores will be equated across different subsets of items (i.e., test forms). An IRT-based PSCC CAT system will provide key patient reported outcome data to help improve patient-centered cancer care and satisfaction for medically underserved populations. Citation Format: Pascal Jean-Pierre, Ying Cheng, Steven Patierno, Peter Raich, Richard Roetzheim, Steven Rosen, Donald Dudley, Karen Freund, Victoria Warren-Mears, Electra Paskett, Kevin Fiscella. Item response theory analysis of the patient satisfaction with cancer-related care: psychometric validation in a multicultural sample of 1,296 participants. [abstract]. In: Proceedings of the 104th Annual Meeting of the American Association for Cancer Research; 2013 Apr 6-10; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2013;73(8 Suppl):Abstract nr 1367. doi:10.1158/1538-7445.AM2013-1367