Item-level Data Research Articles

Development of a General Composite Scale (GENCOMS) for Progressive Neurodegenerative Diseases and Implications for the Assessment of Disease-Modifying Therapies.

The reliable assessment of treatment outcomes for disease-modifying therapies (DMT) in neurodegenerative disease is challenging. The objective of this paper is to describe a generalized framework for developing composite scales that can be applied in diverse, degenerative conditions, termed "GENCOMS." Composite scales optimize the sensitivity for detecting clinically meaningful effects that slow disease progression. The GENCOMS method relies on robust natural history data and/or placebo arm data from DMT trials. Validated scales that are core to the disease process have been identified, and item level data obtained to standardize the response outcomes from 0 (best possible score) to 1 (worst possible score). A partial least squares regression analysis was conducted with temporal change as the dependent variable and change scores in standardized items as the explanatory variables. The derived model coefficients constitute a weighted sum of items that most effectively measure disease progression. The resultant composite scale was optimized to detect disease progression and can be examined in a range of slow or fast progressing populations. The scale can be used in studies with comparable patient populations as an endpoint optimized to measure disease progression and therefore ideally suited to assess treatment effects in DMTs. The methodology presented here provides a generalizable framework for developing composite scales in the assessment of neurodegenerative disease progression and evaluation of DMT effects. By objectively selecting and weighting items from previously validated measures based solely on their sensitivity to disease progression, this methodology allows for the creation of a more responsive measurement of clinical decline. This heightened sensitivity to clinical decline can be utilized to detect modest yet meaningful treatment effects in the early stages of neurogenerative diseases, when it is optimal to begin a DMT.

Validating new symptom emergence as a patient-centric outcome measure for PD clinical trials

IntroductionTracking of emergent symptoms (ES) in de novo Parkinson Disease (PD) patients using Parts Ib and II of the MDS-UPDRS rating scale has been proposed as an outcome measure for PD clinical trials, based on observations in the Safety, Tolerability and Efficacy Assessment of Isradipine for PD (STEADY-PD3) clinical trial. MethodsIndividual item-level data was extracted from the SURE-PD3 study (coded as “PD-1018” in the C-path pooled dataset). We sought to confirm the observations made in the STEADY-PD3 dataset by analyzing data from a different Phase 3 clinical trial, the Phase 3 Study of Urate Elevation in Parkinson Disease (SURE-PD3), in which MDS-UPDRS was assessed more frequently than the 12-month intervals in STEADY-PD3, using similar methodology. ResultsWe were able to broadly validate results that demonstrated the frequency of ES, lack of impact of the introduction of symptomatic medications, and in the reduction in sample size required to demonstrate slowing of disease progression at a group level compared with the traditional total MDS-UPDRS summed score scoring methods. Counts of ES generally correlated modestly with summed MDS-UPRDS scores, both for the various sub-parts and for the overall scale as well. However, instability of individual item responses, especially during the first 6 months of observation complicated the assessment of the temporal evolution and stability of ES over time in the course of the SURE-PD3 study. ConclusionFurther validation using data sets with frequent administration of MDS-UPDRS is necessary to assess value of this approach as an outcome measure in PD clinical trials.

Leveraging machine learning to study how temperament scores predict pre-term birth status

BackgroundPreterm birth (birth at <37 completed weeks gestation) is a significant public heatlh concern worldwide. Important health, and developmental consequences of preterm birth include altered temperament development, with greater dysregulation and distress proneness. AimsThe present study leveraged advanced quantitative techniques, namely machine learning approaches, to discern the contribution of narrowly defined and broadband temperament dimensions to birth status classification (full-term vs. preterm). Along with contributing to the literature addressing temperament of infants born preterm, the present study serves as a methodological demonstration of these innovative statistical techniques. Study designThis study represents a metanalysis conducted with multiple samples (N = 19) including preterm (n = 201) children and (n = 402) born at term, with data combined across investigations to perform classification analyses. SubjectsParticipants included infants born preterm and term-born comparison children, either matched on chronological age or age adjusted for prematurity. Outcome measuresInfant Behavior Questionnaire-Revised Very Short Form (IBQ-R VSF) was completed by mothers, with factor and item-level data considered herein. Results and conclusionsAccuracy estimates were generally similar regardless of the comparison groups. Results indicated a slightly higher accuracy and efficiency for IBQR-VSF item-based models vs. factor-level models. Divergent patterns of feature importance (i.e., the extent to which a factor/item contributed to classification) were observed for the two comparison groups (chronological age vs. adjusted age) using factor-level scores; however, itemized models indicated that the two most critical items were associated with effortful control and negative emotionality regardless of comparison group.

Heterogeneity of Item-Treatment Interactions Masks Complexity and Generalizability in Randomized Controlled Trials

Researchers compare aggregate test data across treatment and control group in randomized controlled trials to determine intervention impact on student achievement. We show that item-level data and analyses can provide information about item-level treatment effect heterogeneity; such findings may be useful for improving interventions and making decisions about outcomes in subsequent studies. We apply Differential Item Functioning techniques to examine variation in the degree to which items show treatment effects. Based on our analysis of 7,244,566 item responses (265,732 students responding to 2,119 items) from 15 RCTs, we find evidence for variation in gains across items. These analyses identify items that are highly sensitive to the interventions—in one case, a single item drives nearly 40% of the observed treatment effect. Our analyses also identifies items that are insensitive and cases wherein treatment effects are consistent across items. We also show that the variation of item-level treatment sensitivity has implications for effect estimate precision. Of the treatments with significant effect estimates, 31% have patterns of item-level treatment sensitivity that may lead to a null effect when considering this source of uncertainty.

Comparison of Patient Health Questionnaire-9, Edinburgh Postnatal Depression Scale and Hospital Anxiety and Depression – Depression subscale scores by administration mode: An individual participant data differential item functioning meta-analysis

Administration mode of patient-reported outcome measures (PROMs) may influence responses. We assessed if Patient Health Questionnaire-9 (PHQ-9), Edinburgh Postnatal Depression Scale (EPDS) and Hospital Anxiety and Depression Scale – Depression subscale (HADS-D) item responses and scores were associated with administration mode. We compared (1) self-administration versus interview-administration; within self-administration (2) research or medical setting versus private; and (3) pen-and-paper versus electronic; and within interview-administration (4) in-person versus phone. We analysed individual participant data meta-analysis datasets with item-level data for the PHQ-9 (N = 34,529), EPDS (N = 16,813), and HADS-D (N = 16,768). We used multiple indicator multiple cause models to assess differential item functioning (DIF) by administration mode. We found statistically significant DIF for most items on all measures due to large samples, but influence on total scores was negligible. In 10 comparisons conducted across the PHQ-9, EPDS, and HADS-D, Pearson's correlations and intraclass correlation coefficients between latent depression symptom scores from models that did or did not account for DIF were between 0.995 and 1.000. Total PHQ-9, EPDS, and HADS-D scores did not differ materially across administration modes. Researcher and clinicians who evaluate depression symptoms with these questionnaires can select administration methods based on patient preferences, feasibility, or cost.

Genetic and phenotypic heterogeneity in early neurodevelopmental traits in the Norwegian Mother, Father and Child Cohort Study

BackgroundAutism and different neurodevelopmental conditions frequently co-occur, as do their symptoms at sub-diagnostic threshold levels. Overlapping traits and shared genetic liability are potential explanations.MethodsIn the population-based Norwegian Mother, Father, and Child Cohort study (MoBa), we leverage item-level data to explore the phenotypic factor structure and genetic architecture underlying neurodevelopmental traits at age 3 years (N = 41,708–58,630) using maternal reports on 76 items assessing children’s motor and language development, social functioning, communication, attention, activity regulation, and flexibility of behaviors and interests.ResultsWe identified 11 latent factors at the phenotypic level. These factors showed associations with diagnoses of autism and other neurodevelopmental conditions. Most shared genetic liabilities with autism, ADHD, and/or schizophrenia. Item-level GWAS revealed trait-specific genetic correlations with autism (items rg range = − 0.27–0.78), ADHD (items rg range = − 0.40–1), and schizophrenia (items rg range = − 0.24–0.34). We find little evidence of common genetic liability across all neurodevelopmental traits but more so for several genetic factors across more specific areas of neurodevelopment, particularly social and communication traits. Some of these factors, such as one capturing prosocial behavior, overlap with factors found in the phenotypic analyses. Other areas, such as motor development, seemed to have more heterogenous etiology, with specific traits showing a less consistent pattern of genetic correlations with each other.ConclusionsThese exploratory findings emphasize the etiological complexity of neurodevelopmental traits at this early age. In particular, diverse associations with neurodevelopmental conditions and genetic heterogeneity could inform follow-up work to identify shared and differentiating factors in the early manifestations of neurodevelopmental traits and their relation to autism and other neurodevelopmental conditions. This in turn could have implications for clinical screening tools and programs.

Identifying the relation between food groups and biological ageing: a data-driven approach.

Heterogeneity in ageing rates drives the need for research into lifestyle secrets of successful agers. Biological age, predicted by epigenetic clocks, has been shown to be a more reliable measure of ageing than chronological age. Dietary habits are known to affect the ageing process. However, much remains to be learnt about specific dietary habits that may directly affect the biological process of ageing. To identify food groups that are directly related to biological ageing, using Copula Graphical Models. We performed a preregistered analysis of 3,990 postmenopausal women from the Women's Health Initiative, based in North America. Biological age acceleration was calculated by the epigenetic clock PhenoAge using whole-blood DNA methylation. Copula Graphical Modelling, a powerful data-driven exploratory tool, was used to examine relations between food groups and biological ageing whilst adjusting for an extensive amount of confounders. Two food group-age acceleration networks were established: one based on the MyPyramid food grouping system and another based on item-level food group data. Intake of eggs, organ meat, sausages, cheese, legumes, starchy vegetables, added sugar and lunch meat was associated with biological age acceleration, whereas intake of peaches/nectarines/plums, poultry, nuts, discretionary oil and solid fat was associated with decelerated ageing. We identified several associations between specific food groups and biological ageing. These findings pave the way for subsequent studies to ascertain causality and magnitude of these relationships, thereby improving the understanding of biological mechanisms underlying the interplay between food groups and biological ageing.

Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity

Analyzing heterogeneous treatment effects (HTEs) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and preintervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment. This study demonstrates that the identical patterns of HTE on test score outcomes can emerge either from variation in treatment effects due to a preintervention participant characteristic or from correlations between treatment effects and item easiness parameters. We demonstrate analytically and through simulation that these two scenarios cannot be distinguished if analysis is based on summary scores alone. We then describe a novel approach that identifies the relevant data-generating process by leveraging item-level data. We apply our approach to a randomized trial of a reading intervention in second grade and show that any apparent HTE by pretest ability is driven by the correlation between treatment effect size and item easiness. Our results highlight the potential of employing measurement principles in causal analysis, beyond their common use in test construction.

Item-level heterogeneous treatment effects of selective serotonin reuptake inhibitors (SSRIs) on depression: implications for inference, generalizability, and identification

Abstract Objectives In analysis of randomized controlled trials (RCTs) with patient-reported outcome measures (PROMs), Item Response Theory (IRT) models that allow for heterogeneity in the treatment effect at the item level merit consideration. These models for “item-level heterogeneous treatment effects” (IL-HTE) can provide more accurate statistical inference, allow researchers to better generalize their results, and resolve critical identification problems in the estimation of interaction effects. In this study, we extend the IL-HTE model to polytomous data and apply the model to determine how the effect of selective serotonin reuptake inhibitors (SSRIs) on depression varies across the items on a depression rating scale. Methods We first conduct a Monte Carlo simulation study to assess the performance of the polytomous IL-HTE model under a range of conditions. We then apply the IL-HTE model to item-level data from 24 RCTs measuring the effect of SSRIs on depression using the 17-item Hamilton Depression Rating Scale (HDRS-17) and estimate heterogeneity by subscale (HDRS-6). Results Our simulation results show that ignoring IL-HTE can yield standard errors that are as much as 50 % too small and create significant bias in treatment by covariate interaction effects when item-specific treatment effects are correlated with item location, and that the application of the IL-HTE model resolves these issues. Our empirical application shows that while the average effect of SSRIs on depression is beneficial (i.e., negative) and statistically significant, there is substantial IL-HTE, with estimates of the standard deviation of item-level effects nearly as large as the average effect. We show that this substantial IL-HTE is driven primarily by systematically larger effects on the HDRS-6 subscale items. Conclusions The IL-HTE model has the potential to provide new insights for the inference, generalizability, and identification of treatment effects in clinical trials using PROMs.

Statistically guided grading judgements: contextualisation or contamination?

ABSTRACT Different sources of assessment evidence are reviewed during International Baccalaureate (IB) grade awarding to convert marks into grades and ensure fair results for students. Qualitative and quantitative evidence are analysed to determine grade boundaries, with statistical evidence weighed against examiner judgement and teachers’ feedback on examinations. A trial was conducted to explore how examiners’ grading decisions were influenced by having access to statistical evidence. Grade awards were replicated in nine exams across five subjects, with examiners accessing all available evidence in one model, and only scripts and grade descriptors in the other. Preliminary findings suggest that both approaches lead to broadly comparable grading outcomes. Focus group feedback indicates that examiners consider judging the grade-worthiness of student work to be their primary role in grade award. Whilst they found item-level data helpful for prioritising questions for review, participants reported that access to evidence such as statistically recommended boundaries can cloud their judgement or encourage strategic grading. This study also raises further questions about the purposes and uses of different forms of statistical evidence, as well as how and when they should be integrated with qualitative evidence in grade awarding.

Clinical Manifestations.

Cognitive decline is most often assessed mean group differences on static measures of cognitive abilities (MacDonald et al., 2009). Another, perhaps more dynamic, method to understanding cognitive functioning is to evaluate variability in performance within an individual, termed intraindividual variability (IIV), which has been associated with reduction in gray matter and less efficient cognitive processing. Using results from a multi-site, longitudinal ARMADA study that validated NIH Toolbox across the cognitive aging spectrum, we investigated whether IIV in reaction time processing can differentiate between older adults with normal cognition, mild cognitive impairment (MCI), or dementia of the Alzheimer type (DAT), as well as the relative magnitude of differences between groups. As part of the Advancing Reliable Measurement in Alzheimer's Disease and cognitive Aging study (ARMADA; Weintraub et al 2020), participants completed the NIH Toolbox Cognition Battery, including two measures of executive functioning, Dimensional Change Card Sort and Flanker Inhibitory Control, which include granular reaction time data in milliseconds. We analyzed standard deviation and mean absolute deviation of item-level data of all trials, correct trials only, and incorrect trials only for three groups: normal controls, those with amnestic MCI, and those with mild DAT. We observe large effect sizes of reaction time by clinical group, with the mild DAT group demonstrating mean standard error across trials of approximately .5 seconds throughout, compared with .25 seconds for MCI and .10 seconds for the control group (Figure 1). This translates to an approximate 5-fold increase in processing speed in the most severe clinical group. Examining time trends, we see that while the NC and MCI group were faster as the test progressed; there is no similar learning effect for the mild DAT group. Similar effect sizes were observed for the Flanker test. This study represents one of the first studies to examine clinical performance using process data from two executive functioning measures on the NIH Toolbox across the cognitive aging spectrum. Future work will explore more definitions of IIV to determine which measure will maximally differentiate between established clinical groups and examine longitudinal differences.

Comparing estimation methods for psychometric networks with ordinal data.

Ordinal data are extremely common in psychological research, with variables often assessed using Likert-type scales that take on only a few values. At the same time, researchers are increasingly fitting network models to ordinal item-level data. Yet very little work has evaluated how network estimation techniques perform when data are ordinal. We use a Monte Carlo simulation to evaluate and compare the performance of three estimation methods applied to either Pearson or polychoric correlations: extended Bayesian information criterion graphical lasso with regularized edge estimates ("EBIC"), Bayesian information criterion model selection with partial correlation edge estimates ("BIC"), and multiple regression with p-value-based edge selection and partial correlation edge estimates ("MR"). We vary the number and distribution of thresholds, distribution of the underlying continuous data, sample size, model size, and network density, and we evaluate results in terms of model structure (sensitivity and false positive rate) and edge weight bias. Our results show that the effect of treating the data as ordinal versus continuous depends primarily on the number of levels in the data, and that estimation performance was affected by the sample size, the shape of the underlying distribution, and the symmetry of underlying thresholds. Furthermore, which estimation method is recommended depends on the research goals: MR methods tended to maximize sensitivity of edge detection, BIC approaches minimized false positives, and either one of these produced accurate edge weight estimates in sufficiently large samples. We identify some particularly difficult combinations of conditions for which no method produces stable results. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Approaches to Item-Level Data with Cross-Classified Structure: An Illustration with Student Evaluation of Teaching

Student evaluation of teaching (SET) questionnaires are ubiquitously applied in higher education institutions in North America for both formative and summative purposes. Data collected from SET questionnaires are usually item-level data with cross-classified structure, which are characterized by multivariate categorical outcomes (i.e., multiple Likert-type items in the questionnaires) and cross-classified structure (i.e., non-nested students and instructors). Recently, a new approach, namely the cross-classified IRT model, was proposed for appropriately handling SET data. To inform researchers in higher education, in this article, the cross-classified IRT model, along with three existing approaches applied in SET studies, including the cross-classified random effects model (CCREM), the multilevel item response theory (MLIRT) model, and a two-step integrated strategy, was reviewed. The strengths and weaknesses of each of the four approaches were also discussed. Additionally, the new and existing approaches were compared through an empirical data analysis and a preliminary simulation study. This article concluded by providing general suggestions to researchers for analyzing SET data and discussing limitations and future research directions.

25 Update to: The Predictive Utility of Various Subjective Cognitive Complaints Using Item Level Data from the Everyday Cognition (ECog) Scales

Objective:Early identification of individuals at risk for dementia provides an opportunity for risk reduction strategies. Many older adults (30-60%) report specific subjective cognitive complaints, which has also been shown to increase risk for dementia. The purpose of this study is to identify whether there are particular types of complaints that are associated with future: 1) progression from a clinical diagnosis of normal to impairment (either Mild Cognitive impairment or dementia) and 2) longitudinal cognitive decline.Participants and Methods:415 cognitively normal older adults were monitored annually for an average of 5 years. Subjective cognitive complaints were measured using the Everyday Cognition Scales (ECog) across multiple cognitive domains (memory, language, visuospatial abilities, planning, organization and divided attention). Cox proportional hazards models were used to assess associations between self-reported ECog items at baseline and progression to impairment. A total of 114 individuals progressed to impairment over an average of 4.9 years (SD=3.4 years, range=0.8-13.8). A subset of individuals (n=352) underwent repeat cognitive assessments for an average of 5.3 years. Mixed effects models with random intercepts and slopes were used to assess associations between baseline ECog items and change in episodic memory or executive function on the Spanish and English Neuropsychological Assessment Scales. Time in years since baseline, the ECog items, and the interaction were key terms of interest in the models. Separate models for both the progression analyses and mixed effects models were fit for each ECog item that included age at the baseline visit, gender, and years of education as covariates.Results:More complaints on five of the eight memory items, three of the nine language items, one of the seven visuospatial items, two of the five planning items, and one of the six organization items were associated with progression to impairment (HR=1.25 to 1.59, ps=0.003 to 0.03). No items from the divided attention domain were significantly associated with progression to impairment. In individuals reporting no difficulty on ECog items at the baseline visit there was no significant change over time in episodic memory(p&gt;0.4). More complaints on seven of the eight memory items, two of the nine language items, and three of the seven visuospatial items were associated with more decline in episodic memory (ps=0.003 to 0.04). No items from the planning, organization, or divided attention domains were significantly associated with episodic memory decline. Among those reporting no difficulty on ECog items at the baseline visit there was slight decline in executive function (ps=&lt;0.001 to 0.06). More complaints on three of the eight memory items and three of the nine language items were associated with decline in executive function (ps=0.002 to 0.047). No items from the visuospatial, planning, organization, or divided attention domains were significantly associated with decline in executive function.Conclusions:These findings suggest that, among cognitively normal older adults at baseline, specific complaints across several cognitive domains are associated with progression to impairment. Complaints in the domains of memory and language are associated with decline in both episodic memory and executive function.

Investigating the feasibility of idiographic network models.

Recent times have seen a call for personalized psychotherapy and tailored communication during treatment, leading to the necessity to model the complex dynamics of mental disorders in a single subject. To this aim, time-series data in one patient can be collected through ecological momentary assessment and analyzed with the graphical vector autoregressive model, estimating temporal and contemporaneous idiographic networks. Idiographic networks graph interindividual processes that may be potentially used to tailor psychotherapy and provide personalized feedback to clients and are regarded as a promising tool for clinical practice. However, the question whether we can reliably estimate them in clinical settings remains unanswered. We conducted a large-scale simulation study in the context of psychopathology, testing the performance of personalized networks with different numbers of time points, percentages of missing data, and estimation methods. Results indicate that sensitivity is low with sample sizes feasible for clinical practice (75 and 100 time points). It seems possible to retrieve the global network structure but not to recover the full network. Estimating temporal networks appears particularly challenging; thus, with 75 and 100 observations, it is advisable to reduce the number of nodes to around six variables. With regard to missing data, full information maximum likelihood and the Kalman filter are effective in addressing random item-level missing data; consequently, planned missingness is a valid method to deal with missing data. We discuss possible methodological and clinical solutions to the challenges raised in this work. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Measurement bias in caregiver-report of early childhood behavior problems across demographic factors in an ECHO-wide diverse sample.

Research and clinical practice rely heavily on caregiver-report measures, such as the Child Behavior Checklist 1.5-5 (CBCL/1.5-5), to gather information about early childhood behavior problems and to screen for child psychopathology. While studies have shown that demographic variables influence caregiver ratings of behavior problems, the extent to which the CBCL/1.5-5 functions equivalently at the item level across diverse samples is unknown. Item-level data of CBCL/1.5-5 from a large sample of young children (N=9087) were drawn from 26 cohorts in the Environmental influences on Child Health Outcomes program. Factor analyses and the alignment method were applied to examine measurement invariance (MI) and differential item functioning (DIF) across child (age, sex, bilingual status, and neurodevelopmental disorders), and caregiver (sex, education level, household income level, depression, and language version administered) characteristics. Child race was examined in sensitivity analyses. Items with the most impactful DIF across child and caregiver groupings were identified for Internalizing, Externalizing, and Total Problems. The robust item sets, excluding the high DIF items, showed good reliability and high correlation with the original Internalizing and Total Problems scales, with lower reliability for Externalizing. Language version of CBCL administration, education level and sex of the caregiver respondent showed the most significant impact on MI, followed by child age. Sensitivity analyses revealed that child race has a unique impact on DIF over and above socioeconomic status. The CBCL/1.5-5, a caregiver-report measure of early childhood behavior problems, showed bias across demographic groups. Robust item sets with less DIF can measure Internalizing and Total Problems equally as well as the full item sets, with slightly lower reliability for Externalizing, and can be crosswalked to the metric of the full item set, enabling calculation of normed T scores based on more robust item sets.

Evaluation of the Electronic Clinical Dementia Rating for Dementia Screening

The Clinical Dementia Rating (CDR) is a well-validated instrument widely used to detect and stage dementia due to Alzheimer disease. The digital Electronic Clinical Dementia Rating (eCDR) can be remotely self-administered and automatically scored, with potential to facilitate efficient dementia screening and staging. To evaluate the association of the eCDR with the CDR and other in-clinic assessments for screening older adults for cognitive impairment. This multisite, cross-sectional study used baseline data from a longitudinal, observational study from 2020 to 2023, including up to 3 years of follow-up. Participants were enrolled from 3 Alzheimer Disease Research Centers and the Brain Health Registry. Participants (aged ≥55 years, with a study partner, and no acute or unstable major medical conditions) were recruited during in-clinic visits or by automated emails. Participants completed the Uniform Data Set Version 3 (UDS; including the CDR) in supervised clinical research settings, and then completed the eCDR remotely, online and unsupervised, using their own device. The primary outcomes were eCDR scores (item; categorical box and global; continuous box and global), CDR scores (item; categorical box and global), and UDS assessment scores. Associations were evaluated using linear and logistic regressions. A total of 3565 participants were contacted, and 288 were enrolled. Among 173 participants with item-level data (mean [SD] age, 70.84 [7.65] years; 76 women [43.9%]), eCDR to CDR concordance was 90% or higher for 33 items (63%) and 70% to 89% for 13 items (25%). Box (domain) level concordance ranged from 80% (memory) to 99% (personal care). The global score concordance rate was 81%. κ statistics were fair to moderate. Among 206 participants with box and global scores (mean [SD] age, 71.34 [7.68] years; 95 women [46.1%]), eCDR continuous global score was associated with CDR global (categorical) score with an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.70-0.87). Correlations between eCDR and in-clinic UDS assessments were similar to those between CDR sum of box scores and the same in-clinic assessments. These findings suggest that the eCDR is valid and has potential use for screening and assessment of older adults for cognitive and functional decline related to Alzheimer disease. Instrument optimization and validation in diverse cohorts in remote settings are crucial for evaluating scalability and eCDR utility in clinical research, trials, and health care settings.

Cross-Classified Item Response Theory Modeling With an Application to Student Evaluation of Teaching

The cross-classified data structure is ubiquitous in education, psychology, and health outcome sciences. In these areas, assessment instruments that are made up of multiple items are frequently used to measure latent constructs. The presence of both the cross-classified structure and multivariate categorical outcomes leads to the so-called item-level data with cross-classified structure. An example of such data structure is the routinely collected student evaluation of teaching (SET) data. Motivated by the lack of research on multilevel IRT modeling with crossed random effects and the need of an approach that can properly handle SET data, this study proposed a cross-classified IRT model, which takes into account both the cross-classified data structure and properties of multiple items in an assessment instrument. A new variant of the Metropolis–Hastings Robbins–Monro (MH-RM) algorithm was introduced to address the computational complexities in estimating the proposed model. A preliminary simulation study was conducted to evaluate the performance of the algorithm for fitting the proposed model to data. The results indicated that model parameters were well recovered. The proposed model was also applied to SET data collected at a large public university to answer empirical research questions. Limitations and future research directions were discussed.

Semantic item-level metrics relate to future memory decline beyond existing cognitive tests in older adults without dementia.

In normal aging, the cognitive domain of semantic memory remains preserved, while the domain of episodic memory declines to some extent. In Alzheimer's disease dementia, both semantic and episodic memory become impaired early in the disease process. Given the need to develop sensitive and accessible cognitive markers for early detection of dementia, we investigated among older adults without dementia whether item-level metrics of semantic fluency related to episodic memory decline above and beyond existing neuropsychological measures and total fluency score. Participants were drawn from the community-based Washington Heights-Inwood Columbia Aging Project cohort (N = 583 English speakers, Mage = 76.3 ± 6.8) followed up to five visits across up to 11 years. We examined the association of semantic fluency metrics with subsequent declines in memory performance using latent growth curve models covaried for age and recruitment wave. Results showed that item-level metrics (e.g., lexical frequency, age of acquisition, and semantic neighborhood density) were associated with a decline in episodic memory-even when covarying for other cognitive tests-while the standard total score was not. Moderation analyses showed that the relationship of semantic fluency metrics with memory decline did not differ across race, sex/gender, or education. In conclusion, item-level data hold a wealth of information with potential to reveal subtle semantic memory impairment, which tracks with episodic memory impairment, among older adults without dementia beyond existing neuropsychological measures. Implementation of psycholinguistic metrics may point to cognitive tools that have better prognostic value or are more sensitive to cognitive change in the context of clinical trials or observational studies. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Young videogamers and their approach to science inquiry

BackgroundWritten instructions seldom need to be read when playing videogames. Instead, gaming often involves early information foraging and expansive exploration behaviors. We use data from the Programme for International Student Assessment (PISA) to explore whether students who regularly play videogames (daily gamers) adopt behaviors that are typical of gaming while they complete a computer-based assessment of science and if such behaviors matter for performance in procedural science knowledge.MethodsWe use item-level data from PISA 2018 from countries that administered the optional ICT questionnaire. Using a sample item and the full set of interactive science items, we develop regression models to estimate differences between daily gamers and other students in information harvesting, time to first action, and procedural science knowledge. We report average associations across countries, country-specific estimates, as well as differences between boys and girls. We report raw differences as well as differences adjusting for background characteristics. To account for the multilevel nature of the data and for the two-stage sampling design, we use replicate weights supplied with the PISA database to estimate robust standard errors using balanced repeated replication (BRR).ResultsIn 2018, 33% 15-year-olds reported playing videogames every day or almost every day. Among boys, that proportion was 49%. Daily gamers do not differ from other students in science content knowledge and in reading fluency. Nevertheless, daily gamers spend marginally less time reading instructions and display more active exploration behaviors in the assessment on items that include simulation tools.ConclusionsScience teachers and assessment developers may find inspiration in games to develop scenarios in which students can practice effective strategies for information harvesting. The extensive exploration of a problem space in order to obtain data in support of future decisions often corresponds to a positive behaviour with multiple advantages in authentic problem situations. By contrast, fast transitions into action may, in the particular situation of an assessment, be an inadequate response. Assessment developers can ensure that instructions are carefully read and understood by test takers and teachers and can guide their students to read instructions adequately.