Classical Item Analysis Research Articles

Social Determinants of Health and Psychological Capital Among Youth Experiencing Homelessness.

Social determinants of health affect health behaviors and outcomes. Youth experiencing homelessness suffer significant deprivation of resources such as inadequate housing, reduced education, poor health care, and decreased economic stability. Inner resources, such as psychological capital, may also be related to health behaviors and health outcomes. In this study, we sought to describe and explore associations among selected determinants of health and self-reported scores on indicators of psychological capital among youth experiencing homelessness. This cross-sectional secondary analysis was conducted with a randomized subsample of 148 youth. We calculated chi-square frequencies to describe the data, classical item analyses to evaluate responses, and correlation tests to examine significance of associations. Youth in this sample demonstrated that they possess inner resources associated with determinants of health. Education, health care, and social support were significantly associated with attributes of psychological capital (hope, efficacy, resilience, optimism). Sexual minority groups had high representation in this subsample (25.7%), indicating a need for more study and equitable services for this population. More research should be conducted to better understand the associations between determinants of health, psychological capital, and health behaviors among disadvantaged youth to advance health equity initiatives.

Medications Non-adherence Reasoning Scale (MedNARS): Development and psychometric properties appraisal.

Proper elucidation of medication non-adherence reasoning especially in older adults might pave the way for an auspicious therapeutic outcome. The main aim of this study was to develop and psychometrically test the Medications Non-adherence Reasoning (MedNARS) questionnaire for application in research and probably practice settings. A mixed methods design was utilized to develop the MedNARS. The item pool was mainly generated based on a qualitative query and literature review. The expert panel approved version of the MedNARS was psychometrically assessed on a convenience sample of 220 older patients with chronic disease. The internal consistency, test-retest reliability, content and face validity of the scale were appraised and its construct validity was assed using exploratory and confirmatory factor analyses. A nine-item version of the MedNARS was drafted based on the classical item analysis procedures and its estimated internal consistency measure of the Cronbach's alpha (0.85) and test-retest reliability (0.96) were in the vicinity of acceptable range. The exploratory factor analysis (EFA) output revealed a unidimensional structure for the MedNARS and the conducted confirmatory factor analysis (CFA) indicated an acceptable data fit for the extracted one-factor model. The goodness of fit indices were as the followings: χ2 /df=1.63(90% CI: 0.02 to 0.11), root mean squared error of approximation (RMSEA)=(0.07), comparative fit index (CFI)=0.95, Tucker-Lewis index (TLI)=0.93 and standardized root mean squared residual (SRMSR)=(0.05). The study findings were indicative of MedNARS's applicability and feasibility for use in assessment of medication non-adherence reasoning among the elderly patients with chronic diseases. The MedNARS as a brief and elder-friendly instrument can be applied both in research and practice settings to enhance efficiency, safety, and health outcomes of the therapeutic recommendations.

Classical Item Analysis from a Signal Detection Perspective

AbstractA conceptualization of multiple‐choice exams in terms of signal detection theory (SDT) leads to simple measures of item difficulty and item discrimination that are closely related to, but also distinct from, those used in classical item analysis (CIA). The theory defines a “true split,” depending on whether or not examinees know an item, and so it provides a basis for using total scores to split item tables, as done in CIA, while also clarifying benefits and limitations of the approach. The SDT item difficulty and discrimination measures differ from those used in CIA in that they explicitly consider the role of distractors and avoid limitations due to range restrictions. A new screening measure is also introduced. The measures are theoretically well‐grounded and are simple to compute by hand calculations or with standard software for choice models; simulations show that they offer advantages over traditional measures.

A Tool for Assessing the Quality of Life of Adolescents in Youth Care: Psychometric Properties of the QOLYSS.

Quality of life (QOL) has gained increased interest as a critical pathway to better understanding the lives and circumstances of children and adolescents in both the general population and among specific populations. Yet, QOL assessment among youngsters in youth care services remains a highly under-researched topic. This study examines the suitability and psychometric properties of a new QOL self-report scale for adolescents between 12 and 18 years old in youth care: the Quality of Life in Youth Services Scale (QOLYSS). The provisional version of the QOLYSS was pre-tested in a sample of 28 adolescents in youth care to examine its applicability and feasibility. Next, a comprehensive evaluation of the psychometric properties of the field-test version was conducted in a sample of 271 adolescents in youth care in Flanders, Belgium (M = 15.43, SD = 1.73). Classical item and factor analyses were carried out per subscale, (test-retest) reliability and item-discriminant validity of the subscales were examined, convergent validity was explored, and confirmatory factor analysis was used to examine the goodness-of-fit of different measurement models. Reliability measures of the scale are satisfactory, results are indicative of convergent validity, and confirmatory factor analysis provides evidence for the eight correlated factors model. Future lines of research concerning the ongoing development and application of the QOLYSS are discussed.

Students Perceived Quality of a Self-Created Mathematics Game-Based Materials

This study determined the quality of games students perceived for the game-based materials they develop after they experience playing a mathematics gameboard. The participants in this study were 88 students in 11th-grade Senior High School composed of 40 students in Sohoton General Academic Strand and 48 students in Kanlanuk Technical-Vocational Livelihood, selected through convenience non-random sampling. Through created games by students, it removed the presence of boredom that improved their performance. It impaired other competencies such as retention and automaticity of information from long-term memory. Moreover, students have developed the competitive, cooperative, and meaningful characteristics of the games, which was considered helpful in improving their knowledge based on how they perceived its' characteristics are important to contain in the game, and their confidence to participate in the lessons. Results from the classical item and test analysis spreadsheet revealed that the prior knowledge of the students increased after the interventions were made and implemented. The characteristics of the game were considered helpful to improve students’ knowledge and confidence in the lessons. These results have implications for in-depth study and analysis to have conclusive results regarding the influence of the intervention in enhancing students’ creativity.

Examining the psychometric properties of the Swedish version of Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) in a clinical sample using classical test theory and item response theory.

The aim of this study was to examine the psychometric properties of the Swedish version of the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) using classical test theory and item response theory (IRT). The CORE-OM is a commonly used 34-item self-report instrument measuring psychological problems/distress covering four domains: subjective well-being, problems/symptoms, functioning and risk. Despite its broad application, only a few studies have used IRT to examine the psychometric properties, and the properties of the Swedish version have only been examined in one initial study. The present study included 1,011 clients with mild to moderate symptoms of distress, applying for psychotherapy at an outpatient training clinic in Sweden. Clients' responses were subjected to classical item analyses as well as IRT (Rasch) analysis using the partial credit model. The classical analyses demonstrated high levels of internal consistency and acceptable levels of item discrimination for the majority of the items, although lower for some items, particularly in the Risk domain. IRT analyses showed that there was a rather good match between item and respondent locations and the measurement precision was high. Disordered step and average measures for some of the items in the Risk domain indicate that these items were problematic from a psychometric point of view and only applicable for a minority of the participants. Differential item functioning for gender in some of the items suggests that they might need to be revised to minimise potential gender bias.

Measuring derivational awareness for Chinese-speaking adolescents

Chinese is a morphologically rich language and converging evidence demonstrates that knowledge of Chinese morphology holds significant relations to several literacy-related skills. However, an area of morphology that has been relatively under-investigated is derivational awareness. Investigations of Chinese derivational morphology have been limited by a lack of technically sound measures. To fill this void, we developed the Chinese Derivational Awareness Measure, which comprises twenty items based on six suffixes. We piloted this measure with Chinese adolescents (n = 174) from seventh and ninth grades to provide reliability and validity evidence. We conducted classical item analysis, nonlinear factor analysis, and bifactor modeling to evaluate the technical qualities of the measure. Results showed that the items were largely adequate based on discrimination indices and various difficulty levels for seventh and ninth grade students. While factor analysis revealed factors related to types of derivational suffixes, bifactor analysis indicated a strongly reliable general factor. These findings support the use of the total score from this newly developed instrument for research on young Chinese learners’ derivational awareness.

Psychometric Analysis of Hypertension Self-Management Behaviors Questionnaire; an Application of Intervention Mapping Approach in Questionnaire Development

Aims: High blood pressure is one of the common main preventable risk factors for many diseases. This study aimed to psychometric properties of the cognitive determinants of hypertension self-management questionnaire among Iranian hypertensive patients based on the Intervention Mapping approach. Instrument & Methods: This psychometric study was conducted in Abadan in 2019. Content Validity Ratio and Content Validity Index of the questionnaire were calculated by ten hypertensive patients and 12 experts. Construct validity was investigated using Classical Item Analysis and Exploratory Factor Analysis by the participation of 315 hypertensive patients. The internal consistency was measured by using Cronbach’s Coefficient Alpha of the various cognitive determinants of the questionnaire. Data were analyzed by SPSS 16. Findings: The mean age of patients was 57.53±11.30 years, ranging from 30 to 70 years. Five factors (attitude, outcome expectation, self-efficacy, subjective norms, and barriers) and behavior were confirmed. The Kaiser-Meyer Olkin test was generally satisfactory (calculated: 0.817). The factors explained 61.86% of the variance of the hypothesized model. Estimated reliability using alpha Cronbach coefficient for each cognitive determinants were as follows: attitude (α=0.75), outcome expectations (α=0.79), self-efficacy (α=0.80), subjective norms (α=0.85), barriers (α=0.88), behavior (α=0.82). Conclusion: The application of the intervention mapping approach in questionnaire development offers a useful questionnaire to measurement self-management hypertension behaviors, and psychometric analysis shows that it could be useful for related planning programs.

Development and validation of the Functional Posttraumatic Cognitions Questionnaire (FPTCQ) in children and adolescents exposed to accidental trauma.

Cognitive models of posttraumatic stress disorder highlight posttraumatic cognitions (PTCs) as a crucial mechanism of trauma adjustment. So far, only dysfunctional PTCs have been investigated in detail. Research on functional PTCs is scarce. This study addresses this gap by developing and validating a self-report measure called Functional Posttraumatic Cognitions Questionnaire (FPTCQ) in children and adolescents. The questionnaire was administered to 114 children and adolescents aged 7 to 15 years who had experienced an acute accidental potentially traumatic event, such as a road traffic accident or a burn injury, and as a result received medical treatment. In addition to classical item analysis and exploratory factor analysis, reliability and construct validity of the FPTCQ were investigated. The exploratory factor analysis revealed a one-factor structure of the FPTCQ. The final 11-item questionnaire displayed satisfactory internal consistency (Cronbach's α = .78), irrespective of age. Functional PTCs were inversely related to dysfunctional PTCs, r = -.44, p < .001, posttraumatic stress symptoms, r = -.35, p < .001, depression symptoms, r = -.22, p < .05, and anxiety symptoms, r = -.43, p < .001, thus supporting construct validity. The FPTCQ is a reliable and valid measure for standardized assessment of functional PTCs among children and adolescents. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

ASSESSING STATISTICS ANXIETY IN AN ONLINE OR HYBRID SETTING: THE ADAPTION AND DEVELOPMENT OF A NEW INSTRUMENT SASOH

With the purpose of developing an instrument for measuring statistics anxiety in the online or hybrid setting, this study tested the newly developed instrument in two stages. Results on item selection and exploratory factor analysis based on pilot testing (n = 115) are presented. Results on classical item analysis, the confirmatory factor analysis, the measurement invariance test results, and the predictive and discriminant validity of the final model based on formal testing (n = 709) are presented. The resulting Statistics Anxiety Scale in the Online or Hybrid setting instrument (SASOH) has 27 items and four dimensions. The four dimensions are Class and Interpretation Anxiety (CI), Fear of Asking for Help Anxiety (FA), Online System Anxiety (OS), and Pre-Conception Anxiety (PC). The results of the confirmatory factor analysis revealed that the four-factor SASOH model represents an adequate description of statistics anxiety in an online or hybrid setting. Moreover, multiple-groups confirmatory factor analysis affirmed that the resulting model achieved at least partial measurement and structural invariance across gender and program. In addition, attitudes toward statistics significantly predicts the four factors of statistics anxiety, and the discriminant validity from mathematics anxiety was confirmed. Recommendations for future studies are also provided.

Development and Validation of a Physics Concept Test in Kinematics for Senior High School Students

Kinematics, a fundamental structure in Mechanics is a critical concept that needs to be realized by students for a more complex analysis of subsequent topics in Physics. One way to determine the effectiveness of Physics teachers in teaching at these trying times is to measure the conceptual understanding of Grade 12-Senior High School (SHS) students in Science, Technology, Engineering, and Mathematics (STEM) track. With the goal of establishing a valid and reliable test questionnaire in Kinematics that can be administered either in a paper-and-pencil approach (asynchronous learning) or online approach (synchronous learning); this study focused on the development and validation process of a 45-item conceptual test in Kinematics. Adhering to the Most Essential Learning Competencies (MELC) set by the Department of Education (DEPED), the initial pool of items was pilot tested using a Google form to 110 SHS students after the items had undergone face and content validation by a panel of experts. Furthermore, Classical Item Analysis by calculating the difficulty and discrimination indices was examined to establish test validity. Reliability analysis was also conducted using Cronbach’s Alpha ( =0.758) and the Kuder-Richardson formula, (KR-20 = 0.761) which resulted in a deletion of 15 items. In general, this Physics concept test in Kinematics showed an acceptable standard of measurement for classroom use which can be utilized by teachers as a form of diagnostic, formative, and summative tests.

An 11-Item Measure of User- and Human-Centered Design for Personal Health Tools (UCD-11): Development and Validation.

BackgroundResearchers developing personal health tools employ a range of approaches to involve prospective users in design and development.ObjectiveThe aim of this paper was to develop a validated measure of the human- or user-centeredness of design and development processes for personal health tools.MethodsWe conducted a psychometric analysis of data from a previous systematic review of the design and development processes of 348 personal health tools. Using a conceptual framework of user-centered design, our team of patients, caregivers, health professionals, tool developers, and researchers analyzed how specific practices in tool design and development might be combined and used as a measure. We prioritized variables according to their importance within the conceptual framework and validated the resultant measure using principal component analysis with Varimax rotation, classical item analysis, and confirmatory factor analysis.ResultsWe retained 11 items in a 3-factor structure explaining 68% of the variance in the data. The Cronbach alpha was .72. Confirmatory factor analysis supported our hypothesis of a latent construct of user-centeredness. Items were whether or not: (1) patient, family, caregiver, or surrogate users were involved in the steps that help tool developers understand users or (2) develop a prototype, (3) asked their opinions, (4) observed using the tool or (5) involved in steps intended to evaluate the tool, (6) the process had 3 or more iterative cycles, (7) changes between cycles were explicitly reported, (8) health professionals were asked their opinion and (9) consulted before the first prototype was developed or (10) between initial and final prototypes, and (11) a panel of other experts was involved.ConclusionsThe User-Centered Design 11-item measure (UCD-11) may be used to quantitatively document the user/human-centeredness of design and development processes of patient-centered tools. By building an evidence base about such processes, we can help ensure that tools are adapted to people who will use them, rather than requiring people to adapt to tools.

Probing the Internal Validity of the LLAMA Language Aptitude Tests

AbstractOver the past decade, the LLAMA language aptitude test battery has come to play an increasingly important role as an instrument in research on individual differences in language development. However, a potentially serious problem that has been pointed out by several scholars is that the LLAMA has not yet been carefully validated. We addressed this issue by examining the internal validity of this test battery. We collected LLAMA data from 350 participants and assessed these data using classical item analysis, Rasch analysis, and principal component analysis within a framework of best practices in educational and psychological test validation. The results show that only one out of the four subtests (LLAMA B) produced scores that fit a latent trait model with sufficient accuracy. This suggests that researchers using the LLAMA battery must treat their results with appropriate carefulness and also that there is potential for refining the LLAMA further.

The preliminary measurement properties of the person-centred community care inventory (PERCCI)

PurposeResearchers investigating person-centredness in older people’s long-term community care are hindered by the lack of appropriate measures. Studies have tended to rely on proxy indicators or generic instruments, risking invalid results. This new research aimed to develop and psychometrically test a person-centredness scale for use in older people’s community services.MethodsQuestionnaire items were sourced from groups of older people and mapped to a conceptual framework of person-centredness. A postal questionnaire in 2015–2016 tested these items with older people supported by mental health and social care services in five areas of England. Dimensionality was assessed through exploratory factor analysis and a confirmatory bifactor model, with classical item analysis removing weak items. Test–retest analysis was undertaken through a repeated postal questionnaire 3 weeks after the first.ResultsThree factors were identified, representing (i) interpersonal and (ii) organisational aspects of person-centred care; and (iii) negatively phrased items. Removing weaker items resulted in an 18-item scale. The bifactor analysis concluded the summary scale was ‘essentially unidimensional’. The Person-centred community care inventory (PERCCI) had excellent reliability, with Intra-Class Correlation Coefficient of 0.886 [95% CI 0.818–0.929]. A priori hypotheses about associations with satisfaction metrics and support variables were broadly confirmed.ConclusionsThe PERCCI has promising measurement properties and can be recommended for use in research with older adults using community mental health and social care services. Future developments must identify how sensitive the instrument is in detecting changing service quality.

Brief assessment of subjective health complaints: Development, validation and population norms of a brief form of the Giessen Subjective Complaints List (GBB-8)

ObjectiveAlthough there is no causal relationship to medical morbidity, routine clinical assessment of somatic symptoms aids medical diagnosis and assessment of treatment effectiveness. Regardless of their causes, somatic symptoms indicate suffering, distress, and help-seeking behavior. The aim of the present study was to develop and validate a brief self-report questionnaire to assess somatic symptom strain. MethodsA brief form of the Giessen Subjective Complaints List (GBB-8) was developed and validated in a large population sample representative of the Federal Republic of Germany (N=2008). Psychometric analyses included confirmation of factor structure, classical item analysis, and measurement invariance tests. The sample furthermore served as a norm group. As indicators of construct validity, correlations with measures of anxiety, depression, alexithymia, and primary care contact were computed. ResultsPsychometric analyses yielded excellent scale properties regarding item characteristics, factor structure, and measurement invariance tests (Cronbach's alpha=0.88; CFI=0.980, TLI=0.965, RMSEA=0.049) for the second-order four-factor model; strict invariance was confirmed for gender, depression status, and physician contacts; strong invariance was confirmed regarding age and age×gender. ConclusionsThe GBB-8 with its four subscales exhaustion, gastrointestinal complaints, musculoskeletal complaints, and cardiovascular complaints proves to be an economic measure of subjective symptom strain. Psychometric analyses deem it suitable for epidemiological research. The availability of norms makes it a potential everyday tool for general practitioners and psychosomatic clinics.

Comparing five depression measures in depressed Chinese patients using item response theory: an examination of item properties, measurement precision and score comparability

BackgroundItem response theory (IRT) has been increasingly applied to patient-reported outcome (PRO) measures. The purpose of this study is to apply IRT to examine item properties (discrimination and severity of depressive symptoms), measurement precision and score comparability across five depression measures, which is the first study of its kind in the Chinese context.MethodsA clinical sample of 207 Hong Kong Chinese outpatients was recruited. Data analyses were performed including classical item analysis, IRT concurrent calibration and IRT true score equating. The IRT assumptions of unidimensionality and local independence were tested respectively using confirmatory factor analysis and chi-square statistics. The IRT linking assumptions of construct similarity, equity and subgroup invariance were also tested. The graded response model was applied to concurrently calibrate all five depression measures in a single IRT run, resulting in the item parameter estimates of these measures being placed onto a single common metric. IRT true score equating was implemented to perform the outcome score linking and construct score concordances so as to link scores from one measure to corresponding scores on another measure for direct comparability.ResultsFindings suggested that (a) symptoms on depressed mood, suicidality and feeling of worthlessness served as the strongest discriminating indicators, and symptoms concerning suicidality, changes in appetite, depressed mood, feeling of worthlessness and psychomotor agitation or retardation reflected high levels of severity in the clinical sample. (b) The five depression measures contributed to various degrees of measurement precision at varied levels of depression. (c) After outcome score linking was performed across the five measures, the cut-off scores led to either consistent or discrepant diagnoses for depression.ConclusionsThe study provides additional evidence regarding the psychometric properties and clinical utility of the five depression measures, offers methodological contributions to the appropriate use of IRT in PRO measures, and helps elucidate cultural variation in depressive symptomatology. The approach of concurrently calibrating and linking multiple PRO measures can be applied to the assessment of PROs other than the depression context.

Psychometric Evaluation and Discussions of English Language Learners’ Listening Comprehension

The importance of listening in the context of English language acquisition is gaining acceptance, but its unique attributes in language performance, while substantively and qualitatively justifiable, are generally not psychometrically defined. This article psychometrically supports listening as a distinct domain among the three other domains of language learning—reading, writing, and speaking—through the analysis of listening data from a large-scale K-12 English language proficiency assessment. Student responses in this study were compared on the basis of cognitive, metacognitive, and other learning strategies between high achieving and low achieving listening groups that were formed using a latent mixture distribution model. Latent mixture distribution and differential items functioning analyses together with classical item analysis were used to substantiate or draw attention to literature findings. The article also highlights the importance of social-interactive items that are inadvertently embedded in an assessment that is primarily created to measure academic language.

Measuring Students’ Motivation: Validity Evidence for the MUSIC Model of Academic Motivation Inventory

This study provides validity evidence for the MUSIC Model of Academic Motivation Inventory (MUSIC Inventory; Jones, 2012), which measures college students’ beliefs related to the five components of the MUSIC Model of Motivation (MUSIC model; Jones, 2009). The MUSIC model is a conceptual framework for five categories of teaching strategies (i.e., eMpowerment, Usefulness, Success, Interest, and Caring) that were derived from research and theory as ones that are critical to students’ motivation ( Jones, 2009). Participants included 338 undergraduate students who provided questionnaire responses in reference to 221 different courses at a large public U.S. university. Our analyses included classical item analysis, confirmatory factor analysis, the calculation of Rasch measurement scales, and Pearson’s correlation coefficients. Results support the validity of scores produced by the MUSIC Inventory for use with college students. This inventory could be useful to instructors and researchers interested in assessing the effects of instruction on students’ motivational beliefs.

The Student Human Papillomavirus Survey: Nurse-Led Instrument Development and Psychometric Testing to Increase Human Papillomavirus Vaccine Series Completion in Young Adults.

The Student Human Papillomavirus Survey (SHPVS) was developed to examine students' perceived benefits or barriers to human papillomavirus (HPV) vaccination. Survey development included (a) 2-phase integrative literature reviews; (b) draft of survey items based on the literature; (c) critique of survey items by young adults, nursing and psychology faculty, and health care providers; and (d) pilot testing. The psychometric properties of the SHPVS were evaluated using classical item analysis and exploratory factor analysis (EFA) among a sample of 527 university students' ages 18-24 years. The estimated Cronbach's alpha for the SHPVS is .74. The SHPVS is a measure of young adults HPV perceived vulnerability, perceived severity, perceived barriers, and perceived benefits of HPV vaccination.

LexMeter: Validation of an Automated System for the Assessment of Lexical Competence of Medical Students as a Prerequisite for the Development of an Adaptive e-learning System

Distance learning is used in medical education, even if some recent meta-analyses indicated that it is no more effective than traditional methods. To exploit the technological capabilities, adaptive distance learning systems aim to bridge the gap between the educational offer and the learner’s need. A decrease of lexical competence has been noted in many western countries, so lexical competence could be a possible target for adaptation. The “Adaptive message learning” project (Am-learning) is aimed at designing and implementing an adaptive e-learning system, driven by lexical competence. The goal of the project is to modulate texts according to the estimated skill of learners, to allow a better comprehension. Lexmeter is the first of the four modules of the Am-learning system. It outlines an initial profile of the learner’s lexical competence and can also produce cloze tests, a test based on a completion task. A validation test of Lexmeter was run on 443 medical students of the 1st, 3rd and 6th year at the University “Sapienza” of Rome. Six cloze tests were automatically produced, with ten gaps each. The tests were different for each year and with varying levels of difficulty. A last cloze test was manually created as a control. The difference of the mean score between the easy tests and the tests with a medium level of difficulty was statistically significant for the 3rd year students but not for 1st and 6th year. The score of the automatically generated tests showed a slight but significant correlation with the control test. The reliability (Cronbach alpha) of the different tests fluctuated under and above .60, as an acceptable level. In fact, classical item analysis revealed that the tests were on the average too simple. Lexical competence is a relevant outcome and its assessment allows an early detection of students at risk. Cloze tests can also be used to assess specific knowledge of technical jargon and to train reasoning skill.