Subtest Reliability Research Articles

Reliability and validity of the post COVID-19 condition stigma questionnaire: a prospective cohort study

Many of the 10-20% percent of COVID-19 survivors who develop Post COVID-19 Condition (PCC, or Long COVID) describe experiences suggestive of stigmatization, a known social determinant of health. Our objective was to develop an instrument, the Post COVID-19 Condition Stigma Questionnaire (PCCSQ), with which to quantify and characterise PCC-related stigma. We conducted a prospective cohort study to assess the reliability and validity of the PCCSQ. Patients referred to our Post COVID-19 Clinic in the Canadian City of Edmonton, Alberta between May 29, 2021 and May 24, 2022 who met inclusion criteria (attending an academic post COVID-19 clinic; age ≥18 years; persistent symptoms and impairment at≥12 weeks since PCR positive acute COVID-19 infection; English-speaking; internet access; consenting) were invited to complete online questionnaires, including the PCCSQ. Analyses were conducted to estimate the instrument's reliability, construct validity, and association with relevant instruments and defined health outcomes. Of the 198 patients invited, 145 (73%) met inclusion criteria and completed usable questionnaires. Total Stigma Score (TSS) on the PCCSQ ranged from 40 to 174/200. The mean (SD) was 103.9 (31.3). Cronbach's alpha was 0.97. Test-retest reliability was 0.92. Factor analysis supported a 6-factor latent construct. Subtest reliabilities were >0.75. Individuals reporting increased TSS occurred across all demographic groups. Increased risk categories included women, white ethnicity, and limited educational opportunities. TSS was positively correlated with symptoms, depression, anxiety, loneliness, reduced self-esteem, thoughts of self-harm, post-COVID functional status, frailty, EQ5D5L score, and number of ED visits. It was negatively correlated with perceived social support, 6-min walk distance, and EQ5D5L global rating. Stigma scores were significantly increased among participants reporting employment status as disabled. Our findings suggested that the PCCSQ is a valid, reliable tool with which to estimate PCC-related stigma. It allows for the identification of patients reporting increased stigma and offers insights into their experiences. The Edmonton Post COVID-19 Clinic is supported by the University of Alberta and Alberta Health Services. No additional sources of funding were involved in the execution of this research study.

Validity and Reliability Study of IST 2000R Indonesian Version

In addition to the limited use of the Intelligenz Struktur Test 2000 Revised (IST 2000R), the adaptation process of the IST 2000R into the Indonesian version is thought to influence this test. Therefore, this study aims to conduct a review of the psychometric properties of the IST 2000R Indonesian version. The research method used is quantitative research and 919 participants, aged 17-30 years, M = 380; F=539 were involved as the sample. The data analysis technique used consisted of the construct validity test using Confirmatory Factor Analysis (CFA) and estimation of internal consistency reliability Cronbach Alpha. Analysis showed that fit results after modification indices with a p-value is 0.05675, RMSEA index is 0.023 ( < 0.05), GFI 0.99, CFI 0.99. and NFI 0.98 ( > 0.9). The results of Composite Reliability (CR) on IST 2000R show that the reliability of sub-tests that measure numerical and abstract intelligence is good, while the reliability of verbal sub-test items is low shown by CR is 0.44.

INTERPRETACE VÝZKUMNÝCH OTÁZEK RELIABILITY JAZYKOVÝCH TESTŮ

This article deals with the topic of reliability of language didactic tests in the framework of language testing. Gained data serve as a means for interpretation and moderation of testing at the language department. Tested students are 1st and 2nd year students of the Faculty of Security Management and the Faculty of Security Management and Security and Legal Studies of the Police Academy of the Czech Republic. For determining the reliability the measurable indicator index of inner consistence with subdivision into scales was set, these scales were used for setting the criteria for the evaluation of the reliability of sub-tests. The reliability of sub-tests is interpreted in a wider contest and is an important step to the optimization of the testing process of creating a test bank for the language department.

Reliabilities of WMS–IV Discrepancy Score Comparisons for Subtests, Indexes, and Selected WAIS–IV Composites

This investigation provides internal consistency reliabilities for the Wechsler Memory Scale–Fourth Edition (WMS–IV) subtest and index discrepancy scores using the standardization samples of the Adult and Older Adult batteries. Subtest reliabilities ranged from .00 to .93 for Adults and .25 to .94 for Older Adults. Three of 91 Adult coefficients and three of 24 Older Adult coefficients reached the recommended level (≥ .90). Index reliabilities ranged from .46 to .94 for 100 Adult Battery comparisons and .19 to 93 for 36 Older Adult Battery comparisons. Each battery had 10 coefficients ≥ .90. Ability–memory discrepancies were reported using WAIS–IV VCI, PRI, WMI, and GAI as intelligence standards. Ability–memory comparisons for the Adult Battery ranged from .82 to .93 with 12 of 20 comparisons ≥ .90; Older Adult Battery reliabilities ranged from .85 to .94 with six of 16 comparisons ≥ .90. In terms of discrepancy score reliabilities, the WMS–IV represents a marked improvement over the WMS–III.

Translation, reliability and validity of Persian version of Adolescent/Adult Sensory Profile in dementia

Our study aimed at the reliability and validity of the Persian version of the Adolescent/Adult Sensory Profile (AASP). This study was conducted on 130 people with dementia as well as 77 healthy elderlies. After translating the AASP into Persian, its content validity was determined based on 9 experts working in the same field. Cronbach’s alpha and the Intraclass correlation coefficient (ICC) were calculated to assess the internal consistency and test–retest reliability of the Persian version of AASP. The two groups of people with dementia and healthy elderlies were compared in terms of scores in the four quadrants by performing an Independent t-test. All items in the Content Validity Ratio (CVR) and Content Validity Index (CVI) received an acceptable score. The Cronbach’s alpha score for different parts of AASP was also calculated (α between 0.894 and 0.916; p < 0.001). The test–retest reliability of sub-tests of AASP was excellent (ICC between 0.885 and 0.948; p < 0.001). There was a statistically significant difference between the healthy Elderlies and dementia persons for low registration, sensory avoiding quadrants (p < 0.05). The Persian version of the AASP questionnaire is a reliable and valid questionnaire for people with dementia in Iran.

The Use of Multivariate Generalizability Theory to Evaluate the Quality of Subscores.

Conventional methods for evaluating the utility of subscores rely on reliability and correlation coefficients. However, correlations can overlook a notable source of variability: variation in subtest means/difficulties. Brennan introduced a reliability index for score profiles based on multivariate generalizability theory, designated as , which is sensitive to variation in subtest difficulty. However, there has been little, if any, research evaluating the properties of this index. A series of simulation experiments, as well as analyses of real data, were conducted to investigate under various conditions of subtest reliability, subtest correlations, and variability in subtest means. Three pilot studies evaluated in the context of a single group of examinees. Results of the pilots indicated that indices were typically low; across the 108 experimental conditions, ranged from .23 to .86, with an overall mean of 0.63. The findings were consistent with previous research, indicating that subscores often do not have interpretive value. Importantly, there were many conditions for which the correlation-based method known as proportion reduction in mean-square error (PRMSE; Haberman, 2006) indicated that subscores were worth reporting, but for which values of fell into the .50s, .60s, and .70s. The main study investigated within the context of score profiles for examinee subgroups. Again, not only indices were generally low, but it was also found that can be sensitive to subgroup differences when PRMSE is not. Analyses of real data and subsequent discussion address how can supplement PRMSE for characterizing the quality of subscores.

Psychometric properties of the Serbian version of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV)

The Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) is an individually administered clinical instrument designed for assessment of intellectual abilities of adolescents and adults. The WAIS-IV consists of 15 subtests (10 core and 5 supplemental) reflecting the efficacy of cognitive functioning in four domains (verbal comprehension-VCI, perceptual reasoning-PRI, working memory-WMI, and processing speed-PSI), and general intellectual ability (Full-Scale IQ-FSIQ). The WAIS-IV was administered to a sample of 262 respondents ? specifically, 104 respondents from sample representative for the wider Belgrade area, 62 schizophrenic, 63 depressive, and 33 patients with intellectual disability. Psychometric properties of WAIS-IV subtests were analysed within the frameworks Item Response Theory (IRT) and Classical Test Theory (CTT). Results show that IRT reliability of subtests ranges from .86 (Comprehension and Picture Completion) to .95 (Cancellation), while the IRT reliability of the FSIQ is .94. Overall, all subtests discriminate participants well along the whole continuum of intellectual abilities. The WAIS-IV enables highly reliable assessment of intellectual abilities on Serbian population.

The Application of Multidimensional Rasch Models in Large Scale Assessment and Validation: An Empirical Example

Introduction. Accountability and diagnostic assessments usually comprise several subtests to provide information on the different components of the abilities tested. For practicality reasons such subtests are usually short. The major drawback of short subtests is their low reliability and imprecision. This study shows how a compensatory multidimensional Rasch model, namely, Multidimensional Random Coefficient Multinomial Logit Model (MRCMLM), can utilize the correlations among subtests to improve estimation and the reliability of measures.Method. The sample was composed of 1021 Iranian high school students. An English language comprehension test composed of four 10-item subtests was adminstretd to the participants. The data were analysed with both MRCMLM and unidimensional Rasch model.Results. Findings showed that subtests that contain only ten items can have the reliability of 45-item tests when students’ abilities are estimated with the MRCMLM. It is also demonstrated that multidimensional analysis gives a better estimate of the true correlations among subtests and can be used as a confirmatory approach to study the componential structure of proficiency constructs.Conclusions. Multidimensional Rasch model can improve the reliability of short subtests. The model can also be used as a confirmatory approach to study the componential structure of proficiency constructs and to show the true association among dimensions of profificency.

Psychometric Properties of the Persian Version of School Function Assessment (SFA) in 6 to 12-Year-Old Children with Physical Disabilities

Background: Nowadays, regarding the increasing awareness of health empowerment models in the field of the nature of the activity, the importance of the participation of children with functional disabilities is emphasized. The purpose of this study is to assess the psychometric properties of the Persian version of school function assessment (SFA). Methods: A group of 80 children with physical disability along with their parents, teachers, and occupational therapists participated in this methodological study. Cronbach’s alpha and the intraclass correlation coefficient (ICC) were calculated to assess the internal consistency and test-retest reliability of Persian version of SFA, respectively. The statistical analysis was done using SPSS software (version 21). Results: The mean age of children in this study was 9.6 years (SD = 1.1). 40 participants were females and 40 were males. Except for a few items, all items had acceptable content validity index (CVI > 0.7) and content validity ratio (CVR > 0.4). The Cronbach’s alpha score for different parts of SFA was also calculated (0.90 < α < 0.98; P < 0.001). The test-retest reliability of sub-tests of SFA was excellent (0.87 < ICC < 0.98; P < 0.001). Conclusion: The Persian version of SFA has a desirable validity and reliability to use in children with physical disabilities and it can be used as a tool for clinical evaluation and physical therapeutic planning by specialists.

In the Netherlands the anti-Flynn effect is a Jensen effect

In this study, 63 observations of secular IQ changes (both Flynn and anti-Flynn effects) are collected from three demographically diverse studies of the Dutch population for the period 1975–2005 (representing the 1950–1990 birth cohorts), along with data on g loadings and subtest reliabilities. The method of correlated vectors is used to explore the association between Flynn and anti-Flynn effect magnitudes, both independently and together, and the g loadings of subtests. Despite a positive vector correlation the Flynn effects are not associated with the Jensen effect (r=.307, ns, N=36), however the anti-Flynn effects are (r=.406, P=.05, N=27). Combined, the vector correlation becomes negative but non-significant (r=−.111, ns, N=63). Declines due to the anti-Flynn effect are estimated at −4.515 points per decade, whereas gains due to the Flynn effect are estimated at 2.175 points per decade. The N-weighted net of these is a loss of −1.350 points per decade, suggesting an overall tendency towards decreasing IQ in the Netherlands with respect to these cohorts. The Jensen effect on the anti-Flynn effect suggests that it may be related to bio-demographic changes within the Netherlands which have reduced ‘genetic-g’, despite the presence of large, parallel gains on subtests that may be relatively more sensitive to cultural-environmental improvements.

Traditional neuropsychological correlates and reliability of the Automated Neuropsychological Assessment Metrics-4 battery for Parkinson’s disease

BackgroundThe Automated Neuropsychological Assessment Metrics Battery-4 for PD offers the promise of a computerized approach to cognitive assessment. MethodsTo assess its utility, the ANAM4-PD was administered to 72 PD patients and 24 controls along with a traditional battery. Reliability was assessed by retesting 26 patients. ResultsThe Cognitive Efficiency Score (CES; a global score) exhibited high reliability (r = 0.86). Constituent variables exhibited lower reliability. The CES correlated strongly with the traditional battery global score, but displayed weaker relationships to UPDRS scores than the traditional score. Multivariate analysis of variance revealed a significant difference between the patient and control groups in ANAM4-PD performance, with three ANAM4-PD tests, Math, Tower, and Pursuit Tracking, displaying sizeable differences. In discriminant analyses these variables were as effective as the total ANAM4-PD in classifying cases designated as impaired based on traditional variables. Principal components analyses uncovered fewer factors in the ANAM4-PD relative to the traditional battery. ANAM4-PD variables correlated at higher levels with traditional motor and processing speed variables than with untimed executive, intellectual or memory variables. ConclusionsThe ANAM4-PD displays high global reliability, but variable subtest reliability. The battery assesses a narrower range of cognitive functions than traditional tests, and discriminates between patients and controls less effectively. Three ANAM4-PD tests, Pursuit Tracking, Math, and Tower performed as well as the total ANAM4-PD in classifying patients as cognitively impaired. These findings could guide the refinement of the ANAM4-PD as an efficient method of screening for mild to moderate cognitive deficits in PD patients.

Reliability of the WAIS-III subtests, indexes, and IQs in individuals with substance abuse disorders.

Reliability of the WAIS-III for 100 male patients with substance abuse disorders was determined. Means for age and education were 46.06 were (SD = 8.81 years) and 12.70 years (SD = 1.51 years). There were 63 Caucasians and 37 African Americans. Split-half coefficients for the 11 subtests (Digit Symbol-Coding, Symbol Search, and Object Assembly were omitted) ranged from .92 for Vocabulary and Digit Span to .77 for Picture Arrangement. The median subtest reliability coefficient was .86. Composite reliabilities were excellent for the Indexes (.94 to .95) and IQs (.94 to .97), with all coefficients > or = .94. Using the Fisher z test to compare correlation coefficients from independent samples, none of the reliability estimates differed significantly from those reported for the WAIS-III standardization sample. Similar findings emerged when reliabilities were determined separately for Caucasian and African American participants.

Psychometric Properties of the Halstead Category Test

Using the Kuder-Richardson formula 20 (KR-20) corrected for item difficulty, the internal consistency reliability of the Halstead Category Test (HCT) was calculated for a sample (N = 398) of normal patients and volunteers, diffusely brain-damaged patients, and undiagnosed patients sent for neuropsychological assessment. The HCT Total Score reliability was .9716 and the subtest reliabilities ranged from .4587 to .9590. An item analysis found that most of the test items, except for those from Subtests I and II, were within established criteria for item difficulty and the discrimination index.

Specificity of the WAIS-R subtests with psychiatric inpatients.

Subtest specificities were determined for each of the 11 WAIS-R subtests for a sample of 200 psychiatric inpatients. The subtest's shared variance (defined as the multiple squared correlation between a subtest and all remaining subtests) was subtracted from reliable variance (defined as the subtest's reliability) to yield a measure of subtest specificity. The Information, Arithmetic, Vocabulary, f1p4rehension, Similarities, Picture Arrangement, and Digit Symbol subtests were judged to have adequate to ample specificity for ipsative interpretation. The Object Assembly, Digit Span, Picture Completion, and Block Design subtests were judged to have inadequate specificity for ipsative interpretation. Implications for the interpretation of WAIS-R results with psychiatric inpatients are discussed.

The calculation of new and improved WISC-III subtest reliability, uniqueness, and general factor characteristic information through the use of data smoothing procedures

Assessment practitioners are often encouraged to adopt an “intelligent” approach to the interpretation of intelligence tests. A fundamental assumption of the “intelligent testing” philosophy is that psychometric test information (e.g., subtest g loadings) should be considered during the interpretive process. The relevant psychometric information is provided in the form of sample-based estimates. Unfortunately, the accuracy of these estimates, and the subsequent qualitative classification of intelligence subtests (e.g., good, fair, poor), are influenced to an unknown degree by sampling error. The current study demonstrated how data smoothing procedures, procedures commonly used in the development of continuous test norms, can be used to provide better estimates of the reliability, uniqueness, and general factor characteristics for the WISC-III subtests. © 1997 John Wiley & Sons, Inc.

The reliability and validity of the learning and study strategies inventory (LASSI) with college developmental students

Abstract This study compared the reliability and validity of the ten subtests of the Learning and Study Strategies Inventory (LASSI) (Weinstein et al., 1987) obtained from 99 college developmental studies students enrolled in an introductory reading course in a southeastern, urban, two‐year community college with the norming values obtained by Weinstein, Schulte, and Palmer (1987). The results indicated that nine of the ten subtest reliability coefficients approached, but did not equal those reported by Weinstein et al. (1987). One‐sample chi‐square analyses also revealed that developmental studies students in the present study scored better than expected on three of the subtests, scored no different than the national norms on four subtests and scored worse than the national norms on three subtests.

WISC-R subtest reliability over time: implications for practice and research.

This study investigated the test-retest reliabilities of the WISC-R IQs and scaled scores on subtests over 3- and 6-yr. intervals for a learning disabled sample. Statistical procedures and individual analyses were conducted on a sample of 107 students who had at least two evaluations. For the total sample, 3-yr. test-retest reliabilities of the Verbal IQ, Performance IQ, and Full Scale IQ were .69, .72, and .77, respectively. Subtest reliabilities ranged from .33 to .71, with a median of .49, but nine subtests had reliabilities of .62 or lower. Individual analyses indicated considerable variability, with some students decreasing as much as 33 IQ points on retest. Similar results were noted for a subsample of 41 students who had three evaluations over a 6-yr. period. Problems with subtest interpretation due to the inconsistency of the WISC-R scores and extensions of this research to the WISC-III are discussed.

New Directions for the Ophthalmic Knowledge Assessment Program (OKAP) Examination

The Ophthalmic Knowledge Assessment Program (OKAP) Task Force of the American Academy of Ophthalmology has reviewed the present OKAP examination and addressed the goals of this resident-in-training examination. Simultaneously, the American Board of Ophthalmology has been assessing its Written Qualifying Examination (WQE) as a certifying tool. The history of the OKAP examination is chronicled along with a discussion of the goals of this examination as a stimulus for resident education. The new OKAP examination is structured specifically for residents-in-training and is now administered separately from the WQE. The OKAP examination is moving in the direction of a criterion referenced examination based on a defined content outline. Twelve subtest areas are tested. Future goals are to improve subtest reliability and increase feedback to residents and program directors.

Assessing different components of reading comprehension: fact or fiction?

The starting point of this study was the debate about the structure of L1 reading comprehension. Kalb, Rabenstein and Rost (1979a; 1979b) claim to assess several different subskills of reading comprehension with the test battery Lesen und Verstehen - Diagnose (Reading and Comprehension- Assessment) (RuC-A) they designed specially for this purpose. To check this, 220 second-grade German elementary-school pupils were tested with the RuC-A. The results show medium-to-high correlations for the eight subtests (0.56 ≤ r ≤ 0.85), and some correlation coefficients are nearly as high as the corresponding subtest reliabilities. Corrected for attenuation, they reach 0.66 ≤ rk ≤ 0.99. Depending on the theoretical perspective, factor analyses (with the reliabilities of the subtests as communalities) yield either one broad factor, 'General reading competence' (accounting for 85% of the nonchance variance) or, at most, two factors, 'inferential reading comprehension' and 'vocabulary' (accounting unrotated for 85% and 6%, rotated for 55% and 36% of the reliable variance). As in other comparable L1 reading compre hension tests, RuC-A apparently cannot measure several clearly distinguish able components of reading comprehension. A reliable and valid diagnosis of typical L1 reading comprehension profiles is not possible.

The Diagnostic Names Task

Increasingly, the field of reading instruction is calling for “authentic”, “meaningful”, or “naturalistic” assessment practices. At the same time, considerable research supports explicit instruction and assessment of decoding skills in reading, especially for low-achieving students. There is obvious need for a naturalistic test of decoding skills that is also valid and reliable, yet no such test yet exists. This study examined some technical qualities of a Diagnostic Names Task which promises to help fill this need. An instrumentation study was conducted with a group of 25 elementary grade disabled readers to answer questions about subtest reliability, sensitivity to change due to instruction, meaningful clustering of subskills, and criterion-related validity. Results were generally encouraging, despite a few sampling and design problems. A large-sample study is recommended next to help further improve subtest reliability.