Evidence Of Criterion Validity Research Articles

The Basic Needs in Games Scale (BANGS): A new tool for investigating positive and negative video game experiences

Players’ basic psychological needs for autonomy, competence, and relatedness are among the most commonly used constructs used in research on what makes video games so engaging, and how they might support or undermine user wellbeing. However, existing measures of basic psychological needs in games have important limitations—they either do not measure need frustration, or measure it in a way that may not be appropriate for the video games domain, they struggle to capture feelings of relatedness in both single- and multiplayer contexts, and they often lack validity evidence for certain contexts (e.g., playtesting vs experience with games as a whole). In this paper, we report on the design and validation of a new measure, the Basic Needs in Games Scale (BANGS), whose 6 subscales cover satisfaction and frustration of each basic psychological need in gaming contexts. The scale was validated and evaluated over five studies with a total of 1246 unique participants. Results supported the theorized structure of the scale and provided evidence for discriminant, convergent and criterion validity. Results also show that the scale performs well over different contexts (including evaluating experiences in a single game session or across various sessions) and over time, supporting measurement invariance. Further improvements to the scale are warranted, as results indicated lower reliability in the autonomy frustration subscale, and a surprising non-significant correlation between relatedness satisfaction and frustration. Despite these minor limitations, BANGS is a reliable and theoretically sound tool for researchers to measure basic needs satisfaction and frustration with a degree of domain validity not previously available.

Measuring Burnout: A Validation Study of The Oldenburg Burnout Inventory for The Sri-Lankan IT Sector

Burnout is a psychological syndrome emerging as a prolonged response to chronic interpersonal stressors on the job. WHO defines burnout as an “occupational phenomenon” and is included in the 11th revision of the International Classification of Diseases (ICD-11). Research reveals the existence of this burnout phenomenon across a range of occupations, such as healthcare professionals, teachers, athletes, pilots, IT professionals etc. A study conducted by Yerbo, a mental wellbeing platform revealed that 2 in 5 of the workers in the IT industry show a high risk of burnout. At present, there are few tools available to measure burnout. After the Maslach Burnout Inventory (MBI), the Oldenburg Burnout Inventory (OLBI) is considered the most widely used measure of burnout and is also freely available. The current study is aimed at determining the measurement validity of the OLBI in the local context based on IT professionals. To determine the content validity of the OLBI, an expert panel was employed. In order to examine the construct validity and concurrent criterion validity, a questionnaire-based study was conducted among 161 (n = 161) randomly chosen IT professionals. MBI and DASS-21 were used to explore the convergent and divergent validity of the OLBI. Four items were deemed not essential to measure burnout based on content validation. The MTMM analysis revealed adequate evidence of the existence of both convergent and divergent validity of the OLBI. The current study was able to provide evidence of content, concurrent criterion and construct related validity of the OLBI in the Sri Lankan context based on IT professionals. These findings suggest that OLBI can be used as a reasonable tool to measure burnout among IT professionals in Sri Lanka. Availability of such a tool will undoubtedly help organizations in ensuring their workforce is healthy and productive. KEYWORDS: Occupational burnout, Content validity, Construct validity, Criterion validity, Oldenburg Burnout Inventory, Multi Trait Multi Method Matrix

Development and preliminary validation of the Clinician Affective REsponse (CARE) scale.

The present study reports on the development and validation of the clinician affective response (CARE) scale. The CARE scale was designed as a self-report measure of therapists' patterns of thoughts, feelings, and behaviors toward the patient during an individual psychotherapy session. An initial pool of 116 items was generated, and its quality was evaluated by subject matter experts. Validation data were gathered from licensed psychotherapists (n=554). We used exploratory factor analysis and item response theory-graded response modeling to select items, confirmatory factor analysis to test how well the factor structure fit the data, and k-fold cross-validation to ascertain the robustness of the model. Criterion validity was evaluated by correlating the scores of the scale with the characteristics of therapists, patients, and treatment. The selected model consists of 15 items and a 3-factor structure, which showed excellent model fit, good internal consistency, and evidence of criterion validity. The CARE scale, short and quick to complete, enables therapists to reflect on and recognize their inner experiences and quantify these experiences in ways conducive to statistical analysis and research. Furthermore, the monitoring of these affective reactions toward their patients can guide therapeutic interventions and inform clinical supervisors.

Initial Validity Evidence for a Measure of Transactional Sex in a U.S. College Student Sample

ABSTRACT The current study provides initial validity evidence for a measure of Transactional Sex (TS). Participants (N = 269) were recruited from a Northeastern University in the United States and consisted of undergraduate and graduate students. Participants completed an online survey through QuestionPro that contained the Transactional Sex Measure (TSM) as well as measures of depression, anxiety, and stress, condom use negotiation self-efficacy and sexual risk, alcohol and drug use, and materialism. Construct and criterion validity were examined. Findings revealed that the TSM provided good criterion validity evidence but the construct validity evidence was minimal. Further studies on the conceptualization of TS and distribution of the TSM across a variety of diverse samples can provide more validity evidence.

Assessing knowledge about self-regulated learning: validation of a measurement tool for preschoolers

IntroductionSelf-regulated learning (SRL), as the self-directed and goal-orientated control of one’s learning process, is an important ability for academic success. Even at preschool age, when its development is at a very early stage, SRL helps to predict later learning outcomes. Valid test instruments are needed to identify preschoolers who require SRL support and help them to start school successfully.MethodsThe present study aimed to provide an adequate SRL test instrument for preschoolers by revising and optimizing an existing strategy knowledge test and validating the revised version–the SRL Strategy Knowledge Test—in a sample of n = 104 German preschoolers (Mage = 5;11 years; 48.1% girls). For the validation, we used measures of (1) SRL and related constructs, (2) psychomotor development, and (3) academic competence, to determine three levels of validity: (a) convergent, (b) divergent, and (c) criterion. All the correlation analyses controlled for child intelligence.ResultsThe results showed that the test is of moderate difficulty and sufficiently reliable (Cronbach’s α = 0.74), can generate normally distributed data, and has a one-factor structure. In line with our hypotheses, we found significant correlations for the convergent and criterion measures, and numerically smaller and insignificant correlations for divergent measures. The correlations for the criterion measures failed to be significant when controlling for intelligence.DiscussionThe missing evidence for criterion validity when controlling for intelligence may have been due to limitations in the measures used to examine criterion validity. The SRL Strategy Knowledge Test can be used in practice to diagnose the need for SRL support and in future studies and interventions on SRL development.

The Kuwaiti-Arabic Trait Emotional Intelligence Questionnaire-Short Form

Abstract: This study aims to cross-culturally adapt and examine the psychometric properties of the Kuwaiti-Arabic Trait Emotional Intelligence-Short Form (TEIQue-SF) through structural equation modeling (SEM). The adapted measure was administered to 1,458 university students in Kuwait together with the Kuwaiti NEO-FFI. Reliability estimates for all TEIQue-SF variables were within the acceptable range, with the exception of certain factors as expected by the literature. SEM results suggested that the bifactor ESEM model fit the data for the TEIQue-SF. Evidence for criterion validity was obtained through relationships between the TEIQue-SF with the Big Five Personality variables. The results suggested that the Kuwaiti-Arabic TEIQue-SF can be considered as a reliable and valid measure to study trait EI with the Kuwaiti population and, consequently, allow for cross-cultural trait EI comparisons.

Detection and Characterization of Endplate Structural Defects on CT: A Diagnostic Accuracy Study.

Diagnostic test study. To determine the reliability and validity or diagnostic accuracy of two previously described endplate structural defect (EPSD) assessment methods. Studies of EPSD may further the understanding of pathoanatomical mechanisms underlying back pain. However, clinical imaging methods used to document EPSD have not been validated, leaving uncertainty about what the observations represent. Using an evaluation manual, 418 endplates on CT sagittal slices obtained from 19 embalmed cadavers (9 men and 10 women, aged 62-91y) were independently assessed by two experienced radiologists and a novice for EPSD using the two methods. The corresponding micro-CT (µCT) from the harvested T7-S1 spines were assessed by another independent rater with excellent intra-rater reliability (Kappa=0.96). Inter-rater reliability was good for presence (Kappa=0.60-0.69) and fair for specific phenotypes (Kappa=0.43-0.58) of EPSD. Erosion, for which the Brayda-Bruno classification lacked a category, was mainly (82.8%) classified as wavy/irregular, while many notched defects (n=15, 46.9%) and Schmorl's nodes (n=45, 79%) were recorded as focal defects using Feng's classification. When compared to µCT, endplate fractures (n=53) and corner defects (n=28) were routinely missed on CT. Endplates classified as wavy/irregular on CT corresponded to erosion (n=29, 21.2%), jagged defects (n=21, 15.3%), calcification (n=19, 13.9%), and other phenotypes on µCT. Some focal defects on CT represented endplate fractures (n=21, 27.6%) on µCT. Overall, with respect to the presence of an EPSD, there was a sensitivity of 70.9% and specificity of 79.1% using Feng's method, and 79.5% and 57.5% using Brayda-Bruno's. Poor to fair inter-rater reliability (k=0.26-0.47) was observed for defect dimensions. There was good inter-rater reliability and evidence of criterion validity supporting assessments of EPSD presence using both methods. However, neither method contained all needed EPSD phenotypes for optimal sensitivity, and specific phenotypes were often misclassified.

The Situational Meaning in Life Evaluation (SMILE): Development and Validation Studies

The present multi-study work presents a new self-report measure of meaning in life in the context of situational life experiences. Study 1 presents the development of the Situational Meaning in Life Evaluation (SMILE), a six-item measure that integrates the three content-dimensions (comprehension, significance and purpose) and the two process-dimensions (presence and search) of meaning in life. The scale is provided with a situational anchor that can be easily adapted to different event- and time-related situations. Two empirical studies examined the psychometric properties of the SMILE measure. Study 2 involved an Italian representative sample of 3035 individuals (51.6% female; Mage= 48.3, range= 18-91, SD= 14.03). Confirmatory Factor Analysis supported the theorized structure of the scale and provided evidence of good internal consistency collected with McDonald’s Omega, generalizability across gender and age was established by measurement invariance, and criterion validity evidence was obtained by correlations with measures of well-being. Study 3 was conducted on a sample of 283 Italian emerging and young adults (76% female; Mage= 26; range= 19-36; SD= 4.09). Results confirmed the SMILE’s structure and internal consistency and added evidence of convergent and incremental validity by conducting a series of hierarchical regressions to test the predictive power of the SMILE over the Meaning in Life Questionnaire on well-being measures. Findings provided evidence of the psychometric properties of the SMILE as a valid and reliable measure of situational meaning in life. Suggestions for future research are discussed.

Psychometric Evaluation of the School Climate and School Identification Measure-Student on Chilean Students: A Bifactor Model Approach.

School climate is a relevant construct for understanding social relations at school. The SCASIM-St has been widely defined as a multidimensional construct; however, new factor structures have not been explored through evidence that allows for interpreting school climate scores from an approach that respects the multidimensionality of the scale and, at the same time, allows for identifying the degree of essential unidimensionality in the data. Consequently, the objective was to analyze the psychometric properties of the SCASIM-St from a bifactor model approach, evaluating the influence of a general school climate factor versus five specific factors. The study involved 1860 students of both sexes (42% males and 58% females), with an average age of 16.63 years (SD = 0.664), from 17 secondary schools in Chile. The results obtained by a confirmatory factor analysis provided evidence that the best model was the bifactor model for the 38 items, with one general factor and five specific factors. The Explained Common Variance (ECV) values and reliability levels by hierarchical omega accounted for a strong general school climate factor with high levels of reliability. Evidence of external criterion validity, assessed through the attitude toward authority scale (AIA-A), showed a theoretically expected and significant relationship between the factors of both instruments. This study confirmed the psychometric robustness of the SCASIM-St scale by means of a bifactor model, allowing for a new, essentially unidimensional interpretation of the scale scores and providing an instrument to measure school climate in Chile.

Evidence of concurrent and predictive criterion validity of the Child Communication Screening Instrument from 0 to 36 months (IRC-36).

To determine evidence of concurrent and predictive criterion validity of the Communication Screening Instrument for children aged 0 to 36 months (IRC-36). 78 parents/guardians of children who attend the childcare service of the Family Health Centers participated in the research, in addition to 33 children aged between 0 and 36 months, invited to the second stage of the study. In its first stage, 13 health professionals were trained to apply the IRC-36 to the children's parents/guardians. In the second moment, the parents responded to a new IRC-36 application, and the children were evaluated with Denver II. IRC-36 correlated with Denver II in more than half of the cases, confirming the instrument's concurrent criterion validity. IRC-36 results in the first stage did not significantly correlate with Denver II. The instrument's cutoff value was 12, which is the reference value between children at risk and not at risk of communication disorders. The instrument had high sensitivity and an accuracy value within the recommended levels. The occurrence of risk of communication changes was higher in the second IRC-36 application. The study presented evidence of concurrent criterion validity, indicating that the instrument has evidence of accuracy and validity measures to screen communication in children aged 0 to 36 months, being able to identify the risk for communication disorders.

Uncovering Cognitive Distortions in Adolescents: Cultural Adaptation and Calibration of an Arabic Version of the “How I Think Questionnaire”

This study adapted and validated the How I Think Questionnaire (HIT-Q), intending to develop an Arabic version of the measure. The study assessed the (a) factorial structure of the Arabic version of the How I Think Questionnaire (A-HIT-Q), (b) construct validity evidence of the A-HIT-Q based on the internal structure of the scale, and (c) criterion validity evidence, highlighting how the cognitive distortions measure relates to some key theoretical variables such as depression. This study involved 762 Tunisian students aged 15–22 years, using a non-probabilistic sampling method. The students were boys (n = 297) and girls (n = 465). They completed self-report forms on Arabic-HIT-Q, depression (HADS), sleep (ISI), and physical activity participation, adhering to all relevant ethical considerations. Exploratory analysis revealed four factors which accounted for 73.46% of the variations in the distortion measure. Reliability analysis showed good internal consistency (α = 0.915) and temporal stability (r = 0.879). Criterion validity evidence showed cognitive distortion (as measured with the A-HIT-Q) was significantly associated with physical activity participation, anxiety, depression, and insomnia. However, no significant relationship has been observed between cognitive distortion, age, gender, and study levels. The evidence gathered supports the utility of the A-HIT-Q. Thus, the instrument demonstrates high efficacy in assessing the levels of cognitive distortions among adolescent students residing in Arabic-speaking regions.

Test of Practical Judgment (TOP-J): Construct, Criterion, and Incremental Validity in a Community Sample of Older Adults.

The Test of Practical Judgment (TOP-J) is a stand-alone judgment measure that is considered to tap into aspects of executive functioning (EF) and inform clinical predictions of daily functioning in older adults. Past validation research is variable and has some limitations. The present study sought to examine the reliability and construct, criterion, and incremental validities of scores on TOP-J 9-item version (TOP-J/9). Participants were 95 community-dwelling older adults aged 60 to 85. Participants completed TOP-J/9, measures of EF and global cognition, and three different modalities of instrumental activities of daily living (IADLs) (self-report, performance-based tasks conducted in the laboratory, and performance-based tasks completed at home over 3weeks). TOP-J/9 scores showed adequate internal consistency (α = 0.73) after correcting for the low number of items. TOP-J/9 was correlated with global cognition and EF, although EF did not survive correction for lower-order processes. Finally, although TOP-J/9 scores were associated with home-based IADL tasks (but not with self-report and laboratory-based IADLs), providing some evidence of criterion validity, they did not incrementally contribute to home-based IADL performance beyond other cognitive measures. However, when two items pertaining to social/ethical judgment were removed, this modified version of TOP-J did relate to EF beyond lower-order processes and contributed uniquely to prediction of home-based IADLs beyond other measures. Results suggest that TOP-J/9 taps into global cognitive status (but not necessarily EF) and predicts "real-world" functioning (but not above and beyond other cognitive measures). TOP-J psychometrics may be improved by removing two social/ethical items.

Scoring Methods for Progress Monitoring Using CBM in Early Writing

This initial exploratory study assessed the evidence of reliability, criterion validity, and growth of scores obtained from total correct word sequences (CWS), correct word sequences per response (CWSR), and a trait-based rubric used with a curriculum-based measure of writing (CBM-W) for progress monitoring. Participants were 26 writers with or at risk for a learning disability (LD) in grades 2-3. Participants were given the CBM-W task twice weekly over 6 weeks and were given the Spelling and Writing Samples subtests of the Woodcock-Johnson Tests of Achievement-III at pre- and posttest. Results indicated large inter-rater reliability coefficients but mixed alternate form reliability and validity evidence. CWS was the only metric to demonstrate significant growth (r = .60). CWS has the strongest evidence for use as a progress monitoring scoring metric for students at risk for LD, although information from CWSR and rubric scores could be instructionally useful.

A coping self-insight scale for adults: development and preliminary psychometric properties

ABSTRACT Background Self-insights focused on the coping process are implicated in the refinement of capacities for resilience. To advance this research, we must identify key coping self-insights and develop a concise measurement tool. Objective The purpose of this paper is to develop evidence for the construct dimensionality and validity of a measure of coping self-insight. Methods Items measuring 13 coping self-insight dimensions were generated via consultation with theoretical work, subject matter experts, and pre-testing items for clarity. Thereafter, the dimensionality of items was assessed with undergraduate students (N = 232) and an online sample (N = 800) via exploratory and confirmatory analyses. Finally, a multi-trait, multi-method approach was used to test discriminant validity in a further sample of students (N = 228). Results The initial item list was reduced to five key dimensions that balanced data-driven and conceptual considerations. Confirmatory Factor Analysis revealed moderate-to-strong correlations (r = .47–.80) among dimensions. We also demonstrated evidence of internal reliability, convergent, criterion, and discriminant validity. Invariance tests for sub-groups of interest (e.g., sex, sample type) frequently demonstrated metric or scalar invariance, except for age sub-groups. Conclusions Findings offer a starting point regarding the types of coping self-insights important for the emergence of resilience and a validated tool for future research.

Psychometric assessment of the US person-centered prenatal and maternity care scales in a low-income predominantly Latinx population in California

ObjectivesTo assess psychometric properties of two scales developed to measure the quality of person-centered care during pregnancy and childbirth in the United States—the Person-Centered Prenatal Care (PCPC-US) and Person-Centered Maternity Care (PCMC-US) scales—in a low-income predominantly Latinx population in California.MethodsData were collected from July 2020 to June 2023 from surveys of low-income pregnant and birthing people in Fresno, California, participating in the “Engaging Mothers and Babies; Reimagining Antenatal Care for Everyone” (EMBRACE) trial. Research staff administered the 26-item PCPC-US scale at 30–34 weeks’ gestation (n = 315) and the 35-item PCMC-US scale at 10–14 weeks after birth (n = 286), using the language preferred by the participant (English or Spanish). We assessed construct, criterion, and known group validity and internal consistency of the scales.Results78% of respondents identified as Latinx. Factor analysis identified one dominant factor for each scale that accounted for over 60% of the cumulative variance, with most items loading at > 0.3. The items also loaded adequately on sub-scales for “dignity and respect,” “communication and autonomy,” and “responsive and supportive care.” Cronbach’s alpha for the full scales were > 0.9 and between 0.70 and 0.87 for the sub-scales. Summative scores range from 0 to 100, with higher scores indicating higher person-centered care. Correlations with scores on scales measuring prenatal care quality and birth experience provided evidence for criterion validity, while associations with known predictors provided evidence for known-group validity.ConclusionsThe PCPC-US and PCMC-US scales, which were developed using a community-engaged process and found to have good psychometric properties in a largely high-income sample of Black women, were shown to also have good psychometric properties in a sample of low-income primarily Latinx women. Both scales provide valid and reliable tools to measure person-centered care experiences among minoritized communities to support efforts to reduce existing birth inequities.

Symposium 5

While magnetic resonance imaging (MRI) is the most established imaging modality to assess autoimmune myositis (AIM), there is a need for less expensive and more accessible tools. To analyze whether there is sufficient data from published literature to demonstrate that ultrasound presents good metric properties (truth, discrimination and feasibility) in AIM, the OMERACT ultrasound group for myositis performed a systematic literature review. The inclusion criteria required original research involving adult humans, reported in English, assessing ultrasound and elastography in patients with an AIM. Conference abstracts and computer-assisted diagnostics that focused on technique and not ultrasound domains were excluded. Approximately 2670 articles were identified. 41 full-text articles were included in the final analysis. There were 551 AIM patients studied. Eighteen studies (43.9%) had a control group, of which 15 (63.3%) were healthy controls. The age of participants (including controls) varied from 18 to 86 years, and most were females (59%). Diagnosis of AIM was largely biopsy-proven, although some were derived through clinical presentation, positive clinical imaging (ultrasound or otherwise) and/or electromyography and steroid responsiveness. The features examined with ultrasound in the 41 included articles consisted of: muscle echogenicity, bulk, atrophy, architecture, power Doppler, perfusion characteristics, shear wave modulus, shear wave velocity, elasticity index and fasciculations. 12 studies (29.2%) used quantitative methods to assess these characteristics, whilst others used semi-quantitative, dichotomous/binary and descriptive scoring systems. Criterion validity was met in 14 studies (12/14, 85.7%) and construct validity in 22 studies (22/25, 88.0%). Most published articles reported Level 3b to Level 5 evidence with varying degrees of bias. There was only one longitudinal study examining discrimination. Reliability and feasibility were under-reported. This was the first systematic review studying the utility of ultrasound in AIM, which suggested some evidence for criterion and construct validity. Further validation processes are underway.

3 Stricker Learning Span criterion validity: remote self-administration of a computer adaptive word list memory test shows similar ability to differentiate PET-defined biomarker groups as in-person Rey Auditory Verbal Learning Test performance in cognitively unimpaired individuals on the Alzheimer’s continuum

Objective:The Stricker Learning Span (SLS) is a computer-adaptive word list memory test specifically designed for remote assessment and self-administration on a web-based multi-device platform (Mayo Test Drive). Given recent evidence suggesting the prominence of learning impairment in preclinical Alzheimer’s disease (AD), the SLS places greater emphasis on learning than delayed memory compared to traditional word list memory tests (see Stricker et al., Neuropsychology in press for review and test details). The primary study aim was to establish criterion validity of the SLS by comparing the ability of the remotely-administered SLS and inperson administered Rey Auditory Verbal Learning Test (AVLT) to differentiate biomarkerdefined groups in cognitively unimpaired (CU) individuals on the Alzheimer’s continuum.Participants and Methods:Mayo Clinic Study of Aging CU participants (N=319; mean age=71, SD=11; mean education=16, SD=2; 47% female) completed a brief remote cognitive assessment (∼0.5 months from in-person visit). Brain amyloid and brain tau PET scans were available within 3 years. Overlapping groups were formed for 1) those on the Alzheimer’s disease (AD) continuum (A+, n=110) or not (A-, n=209), and for 2) those with biological AD (A+T+, n=43) vs no evidence of AD pathology (A-T-, n=181). Primary neuropsychological outcome variables were sum of trials for both the SLS and AVLT. Secondary outcome variables examined comparability of learning (1-5 total) and delay performances. Linear model ANOVAs were used to investigate biomarker subgroup differences and Hedge’s G effect sizes were derived, with and without adjusting for demographic variables (age, education, sex).Results:Both SLS and AVLT performances were worse in the biomarker positive relative to biomarker negative groups (unadjusted p’s<.05). Because biomarker positive groups were significantly older than biomarker negative groups, group differences were attenuated after adjusting for demographic variables, but SLS remained significant for A+ vs A- and for A+T+ vs A-T- comparisons (adjusted p’s<.05) and AVLT approached significance (p’s .05-.10). The effect sizes for the SLS were slightly better (qualitatively, no statistical comparison) for separating biomarker-defined CU groups in comparison to AVLT. For A+ vs A- and A+T+ vs A-T- comparisons, unadjusted effect sizes for SLS were -0.53 and -0.81 and for AVLT were -0.47 and -0.61, respectively; adjusted effect sizes for SLS were -0.25 and -0.42 and for AVLT were -0.19 and -0.26, respectively. In secondary analyses, learning and delay variables were similar in terms of ability to separate biomarker groups. For example, unadjusted effect sizes for SLS learning (-.80) was similar to SLS delay (.76), and AVLT learning (-.58) was similar to AVLT 30-minute delay (-.55) for the A+T+ vs AT- comparison.Conclusions:Remotely administered SLS performed similarly to the in-person-administered AVLT in its ability to separate biomarker-defined groups in CU individuals, providing evidence of criterion validity. The SLS showed significantly worse performance in A+ and A+T+ groups (relative to A- and A-T-groups) in this CU sample after demographic adjustment, suggesting potential sensitivity to detecting transitional cognitive decline in preclinical AD. Measures emphasizing learning should be given equal consideration as measures of delayed memory in AD-focused studies, particularly in the preclinical phase.

71 Examining the Psychometric Validity of NeuroScreen to Assess Neurocognition in Hospitalized Psychosis Patients in Uganda

Objective:People with psychotic disorders often experience neurocognitive deficits, such as neurocognitive impairment (NCI), which can negatively affect their daily activities (e.g., performing independent tasks) and recovery. Because of this, the American Psychology Association advocates integrating neurocognitive testing into routine care for people living with psychotic disorders, especially those in their first episode, to inform treatment and improve clinical outcomes. However, in low-and-middle income countries (LMICs), such as Uganda where the current study took place, administering neurocognitive tests in healthcare settings presents numerous challenges. In Uganda there are few resources (e.g., trained clinical staff, and culturally relevant and normed tests) to routinely offer testing in healthcare settings. NeuroScreen is a brief, highly automated, tablet-based neurocognitive testing tool that can be administered by all levels of healthcare staff and has been translated into indigenous Ugandan languages. To examine the psychometric properties of NeuroScreen, we measured convergent and criterion validity of the NeuroScreen tests by comparing performance on them to performance on a traditional battery of neurocognitive tests widely used to assess neurocognition in people with psychotic disorders, the Matric Consensus Cognitive Battery (MCCB).Participants and Methods:Sixty-five patients admitted into Butabika Mental Referral Hospital in Uganda after experiencing a psychotic episode and forty-seven demographically similar control participants completed two neurocognitive test batteries: the MCCB and NeuroScreen. Both batteries include tests measuring the neurocognitive domains of executive functioning, working memory, verbal learning, and processing speed. Prior to completing each battery, patients were medically stabilized and could not exhibit any positive symptoms on the day of testing. On the day of testing, medication dosages were scheduled so that patients would not experience sedative effects while testing. To examine convergent validity, we examined correlations between overall performance on NeuroScreen and the MCCB, as well as tests that measured the same neurocognitive domains. To examine criterion validity, an ROC curve was computed to examine the sensitivity and specificity of NeuroScreen to detect NCI as defined by the MCCB.Results:There was a large correlation between overall performance on NeuroScreen and the MCCB battery of tests, r(110) = .65, p < .001. Correlations of various strengths were found among tests measuring the same neurocognitive domains in each battery: executive functioning [r(110) = .56 p <.001], processing speed [r(110) = .44, p <.001)], working memory [r(110) = .29, p<.01], and verbal learning [r(110) = .22, p < .01]. ROC analysis of the ability of NeuroScreen to detect MCCB defined NCI showed an area under curve of .798 and optimal sensitivity and specificity of 83% and 60%, respectively.Conclusions:Overall test performance between the NeuroScreen and MCCB test batteries was similar in this sample of Ugandans with and without a psychotic disorder, with the strongest correlations in tests of executive functioning and processing speed. ROC analysis provided criterion validity evidence of NeuroScreen to detect MCCB defined NCI. These results provide support for use of NeuroScreen to assess neurocognitive functioning among patients with psychotic disorders in Uganda, however more work needs to be to determine how well it can be implemented in this setting. Future directions include assessing cultural acceptability of NeuroScreen and generating normative data from a larger population of Ugandan test-takers.

Intelligibility in Context Scale: Psychometric evidence and implications for Saudi Arabic-English-speaking preschoolers

Purpose: To evaluate the reliability, validity, and diagnostic accuracy (sensitivity, specificity) of the Intelligibility in Context Scale in Saudi Arabic (ICS-SA) and English (ICS-E) as potential measures of functional speech intelligibility in bilingual preschool-aged Saudi Arabian children. Method: The study included 36 parent-child (aged 3;0 to 6;5 years; months) dyads who were bilingual speakers of Saudi Arabic and English, which included two groups—typically developing (TD, n = 29) and suspected speech sound disordered (sSSD, n = 7). The children’s intelligibility was evaluated in both languages using the ICS-SA and ICS-E. Children’s single-word productions were collected in both languages and calculated for percentage of consonants, vowels, and phonemes correct (i.e. PCC/PVC/PPC) to establish their speech-sound competence. Result: The mean scores for the entire sample (n = 36) were 4.43 (SD = 0.79) for the ICS-SA and 4.48 (SD = 0.79) for the ICS-E, showing that parents rated their children’s intelligibility in both languages similarly. Both the ICS-SA and ICS-E demonstrated excellent internal consistency (α = 0.96 and α = 0.95, respectively). There was high inter-rater and test-retest reliability for the ICS-SA, while there was fair to high inter-rater and test-retest reliability in ICS-E. Significant correlations were found for the ICS-SA, but weak correlations were noted for the ICS-E, which indicated fair to moderate evidence of criterion validity. Construct validity efforts indicated a weak correlation with age on both the ICS-SA and ICS-E. The findings also indicated high discriminant accuracy for both the ICS-SA (0.86, 0.86) and ICS-E (0.71, 0.69). Conclusion: This study provides initial validation and reliability evidence for using the ICS-SA and ICS-E with Saudi Arabic- and English-speaking preschoolers, however, ICS-E scores suggest further testing is warranted. By extension, these findings expand the bilingual knowledge base and offer new tools for identifying children in Saudi Arabia who may be at risk for having a speech sound disorder.

Stricker Learning Span criterion validity: a remote self-administered multi-device compatible digital word list memory measure shows similar ability to differentiate amyloid and tau PET-defined biomarker groups as in-person Auditory Verbal Learning Test.

The Stricker Learning Span (SLS) is a computer-adaptive digital word list memory test specifically designed for remote assessment and self-administration on a web-based multi-device platform (Mayo Test Drive). We aimed to establish criterion validity of the SLS by comparing its ability to differentiate biomarker-defined groups to the person-administered Rey's Auditory Verbal Learning Test (AVLT). Participants (N = 353; mean age = 71, SD = 11; 93% cognitively unimpaired [CU]) completed the AVLT during an in-person visit, the SLS remotely (within 3 months) and had brain amyloid and tau PET scans available (within 3 years). Overlapping groups were formed for 1) those on the Alzheimer's disease (AD) continuum (amyloid PET positive, A+, n = 125) or not (A-, n = 228), and those with biological AD (amyloid and tau PET positive, A+T+, n = 55) vs no evidence of AD pathology (A-T-, n = 195). Analyses were repeated among CU participants only. The SLS and AVLT showed similar ability to differentiate biomarker-defined groups when comparing AUROCs (p's > .05). In logistic regression models, SLS contributed significantly to predicting biomarker group beyond age, education, and sex, including when limited to CU participants. Medium (A- vs A+) to large (A-T- vs A+T+) unadjusted effect sizes were observed for both SLS and AVLT. Learning and delay variables were similar in terms of ability to separate biomarker groups. Remotely administered SLS performed similarly to in-person-administered AVLT in its ability to separate biomarker-defined groups, providing evidence of criterion validity. Results suggest the SLS may be sensitive to detecting subtle objective cognitive decline in preclinical AD.