Computerized Adaptive Testing Research Articles

Efficiency, Precision, Validity, and Reliability of GlauCAT-Asian Computerized Adaptive Tests in Measuring Glaucoma-Related Quality of Life.

To determine the efficiency, precision, and agreement of GlauCAT-Asian and its corresponding validity and reliability. In this cross-sectional study, 219 participants (mean ± standard deviation age, 66.59 ± 8.61years; 34% female) across the spectrum of glaucoma severity and 50 glaucoma suspects were recruited from glaucoma clinics in Singapore. Participants answered seven computerized adaptive testing (CAT) evaluations (Ocular Comfort, Activity Limitation, Lighting, Mobility, Concerns, Psychosocial, Glaucoma Management) and underwent eye examinations. Efficiency (mean number of items required for each CAT and time taken for CAT versus full item banks [IBs]), agreement (concordance between CATs and full IB person measures, henceforth referred to as scores), and precision (standard error of measurement [SE]) were evaluated. Other validity and reliability metrics were also assessed. The mean number of items administered ranged from 9 (Mobility/Glaucoma Management) to 12 (Ocular Comfort). Compared to answering the full IBs, CATs provided an average time saving of 38.3% (range, 10% to 70.6% for Lighting and Activity Limitation, respectively). Agreement between scores obtained by CAT versus full IB was high (intracorrelation coefficient ≥0.75), as was precision of score estimates (mean SE range: 0.35 for Psychosocial to 0.29 for Mobility). Scores from Activity Limitation, Mobility, Lighting, and Concerns decreased significantly as glaucoma severity increased (criterion validity; P-trend <0.05). All tests displayed good convergent/divergent validity and test-retest reliability. GlauCAT-Asian provides efficient, precise, accurate, valid, and reliable measurement of the patient-centered impact of glaucoma. GlauCAT-Asian may provide a valuable clinical tool for ophthalmologists to monitor impact of disease progression and the effectiveness of therapies.

Differential Performance of Computerized Adaptive Testing in Students With and Without Disabilities – A Simulation Study

Technologies that enable individualization for students have significant potential in special education. Computerized Adaptive Testing (CAT) refers to digital assessments that automatically adjust their difficulty level based on students' abilities, allowing for personalized, efficient, and accurate measurement. This article examines whether CAT performs differently for students with and without special educational needs (SEN). Two simulation studies were conducted using a sample of 709 third-grade students from general and special schools in Germany, who took a reading test. The results indicate that students with SEN were assessed with fewer items, reduced bias, and higher accuracy compared to students without SEN. However, measurement accuracy decreased, and test length increased for students whose abilities deviated more than two SD from the norm. We discuss potential adaptations of CAT for students with SEN in the classroom, as well as the integration of CAT with AI-supported feedback and tailored exercises within a digital learning environment.

Post-Intensive Care Sequelae in Pediatrics-Results of an Early Rehabilitation Implementation Study.

To compare post-PICU discharge functioning, health-related quality of life (HRQL), and parental stress before and after the implementation of an early rehabilitation bundle. Prospective cohort substudy within an early rehabilitation implementation program, conducted at the PICUs at McMaster Children's Hospital and London Health Sciences, London, Ontario, Canada. A bundle consisting of: 1) analgesia-first sedation; 2) delirium monitoring and prevention; and 3) early mobilization. Patients with an anticipated 48-hour PICU length of stay were approached for consent to participate. Critically ill children with an anticipated 48-hour PICU length of stay were approached for consent to participate. Patient-/proxy-reported outcome measures were assessed at baseline, PICU discharge, and 1 and 3 months post-PICU discharge using: 1) Pediatric Evaluation of Disability Inventory Computer Adaptive Test to assess physical, social, cognitive, and responsibility/caregiver domains of functioning; 2) KIDSCREEN to assess HRQL; and 3) the Pediatric Inventory for Parents to assess caregiver stress. A total of 117 participants were enrolled. Patient demographic characteristics were similar in the pre- and post-intervention groups. Following bundle implementation, 30 of 47 respondents (63.8%) experienced functional decline and 18 of 45 (40%) experienced low HRQL at PICU discharge. Eighteen of 36 (50%) at 1 month and 14 of 38 (36.8%) at 3 months experienced either persistent functional decline and/or low HRQL; 2.8% and 2.6% at 1- and 3-month follow-up, respectively, experienced both persistent functional decline and low HRQL. There were no significant differences in the rates of persistent functional decline, low HRQL, or caregiver stress scores post-bundle compared with pre-rehabilitation bundle implementation. We were unable to adequately determine the efficacy of a rehabilitation bundle on patient-centered outcomes as this substudy was not powered for these outcomes. Our results did reveal that persistent low functioning is common in PICU survivors, more common than low HRQL, while experiencing both functional decline and low HRQL was uncommon.

Patient Reported Clinical Outcomes Following PCL Suture Augmentation in Patients with Multiligamentous Knee Injury: A Retrospective Observational Study.

To compare the patient-reported outcomes between patients with posterior cruciate ligament (PCL) reconstruction or repair alone versus PCL reconstruction or repair with internal bracing (IB) in the context of multi-ligament knee injuries (MLKI). All patients who underwent surgical management of MLKI at two institutions between 2006 and 2020 were retrospectively identified and offered participation in the study. Patient reported outcomes were measured via three instruments: Lysholm Knee score, Multiligament Quality of Life (ML-QOL), and the Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive testing (CAT). The postoperative outcomes and reoperation rates were compared between the internal bracing and non-internal bracing groups. Fifty-two patients were analyzed; 34 were included in the IB group (17.6% female; age 33.1 ±1.60 years), and 18 were included in the non-IB group (11.1% female; age 34.1 ±3.72 years). Mean follow-up time of the entire cohort was 1.44 ± 0.22 years (IB: 1.21 ± 0.18; non-IB: 2.1 ±0.65). There were no significant differences between PROMIS CAT [PROMIS Pain (54.4 ±1.78 vs 51.7 ±1.70, p=0.319), Physical Function (44.3 ±2.27 vs 47.9 ±1.52, p=0.294), Mobility (44.0 ±1.71 vs 46.1 ±2.10, p=0.463)], ML-QOL [ML-QOL Physical Impairment (40.7 ±4.21 vs 41.7±5.10, p=0.884), Emotional Impairment (49.2 ±4.88 vs 44.7±5.87, p=0.579), Activity Limitation (43.5 ±4.56 vs 31.5±3.62, p=0.087), Societal Involvement (44.9 ±4.96 vs 37.5 ±5.30, p=0.345)] and Lysholm knee score (61.8 ±4.55 vs 61.0 ±4.95, p=0.916) postoperatively compared to the non-IB group. In this group of patients, function and patient-reported outcomes between patients treated with PCL reconstruction and repair without internal brace versus those with additional internal brace augmentation were not significantly different. Further research encompassing a larger patient sample is necessary to investigate the efficacy of the internal brace for PCL injury in the context of MLKI injuries.

The Paradigm Shift in Education in Big Data Era: Exploring the Intersection of Historical Assessment Frameworks and AI-Powered Assessment Methods in Education

This paper delves into the dynamic intersection between traditional assessment processes and AI-powered technology assessment models within the expansive landscape of the big data era. Tracing the origins of large-scale standardized testing, from Francis Galton's pioneering intelligence testing proposals to the subsequent development of norm-referenced testing (NRT) by Alfred Binet and Theodore Simon, the historical trajectory underscores the evolution towards authentic assessment paradigms such as computerized adaptive testing (CAT) and performance-based assessment (PBA). These shifts reflect a broader aim of equipping educators with tools to measure student learning effectively and foster continuous learning beyond the confines of the classroom. However, despite the proliferation of methods integrating big data and AI technologies in educational settings, a discernible gap persists between technological capabilities and their practical implementation in education, notably in the realm of student assessment and evaluation. This gap highlights challenges in enhancing assessment accuracy and reliability, evaluating non-cognitive skills, and delivering personalized feedback. The introduction of AI-powered technologies like ChatGPT raises ethical considerations regarding fairness, privacy, and bias, necessitating responsible and equitable deployment.This paper addresses fundamental research questions concerning the historical influences on assessment frameworks, the role of AI-powered technology in driving paradigm shifts, and the ethical considerations surrounding its application. Through a comprehensive literature review, it explores the potential of AI technology to enhance assessment accuracy and reliability while advocating for its responsible use. Despite the challenges posed by traditional assessment processes and AI-powered technology assessment models, educators continually strive to improve student learning and readiness for lifelong learning.By celebrating the invaluable contributions of educators, we can shift the narrative surrounding assessment practices towards a more positive and supportive environment, fostering ethical and innovative uses of AI-powered technology. This celebration serves as inspiration for future research and development in assessment, emphasizing the importance of building upon educators’ innovative practices to shape the future of assessment in the big data era.

Computerized adaptive testing to screen pre-school children for emotional and behavioral problems.

Questionnaires to detect emotional and behavioral (EB) problems in preventive child healthcare (PCH) should be short; this potentially affects their validity and reliability. Computerized adaptive testing (CAT) could overcome this weakness. The aim of this study was to (1) develop a CAT to measure EB problems among pre-school children and (2) assess the efficiency and validity of this CAT. We used a Dutch national dataset obtained from parents of pre-school children undergoing a well-child care assessment by PCH (n = 2192, response 70%). Data regarded 197 items on EB problems, based on four questionnaires, the Strengths and Difficulties Questionnaire (SDQ), the Child Behavior Checklist (CBCL), the Ages and Stages Questionnaire: Social Emotional (ASQ:SE), and the Brief Infant-Toddler Social and Emotional Assessment (BITSEA). Using 80% of the sample, we calculated item parameters necessary for a CAT and defined a cutoff for EB problems. With the remaining part of the sample, we used simulation techniques to determine the validity and efficiency of this CAT, using as criterion a total clinical score on the CBCL. Item criteria were met by 193 items. This CAT needed, on average, 16 items to identify children with EB problems. Sensitivity and specificity compared to a clinical score on the CBCL were 0.89 and 0.91, respectively, for total problems; 0.80 and 0.93 for emotional problems; and 0.94 and 0.91 for behavioral problems. Conclusion: A CAT is very promising for the identification of EB problems in pre-school children, as it seems to yield an efficient, yet high-quality identification. This conclusion should be confirmed by real-life administration of this CAT. What is Known: • Studies indicate the validity of using computerized adaptive test (CAT) applications to identify emotional and behavioral problems in school-aged children. • Evidence is as yet limited on whether CAT applications can also be used with pre-school children. What is New: • The results of this study show that a computerized adaptive test is very promising for the identification of emotional and behavior problems in pre-school children, as it appears to yield an efficient and high-quality identification.

Students' and lecturers' perceptions of computerised adaptive testing as the future of assessing students

The Covid-19 pandemic has been a catalyst for the increased adoption and acceptance of technology in teaching and learning, forcing higher education institutions (HEIs) to re-look at their assessment strategy as learning, development, and engagement move more fluidly into the online arena. A total of 623 lecturers and students in private HEIs in South Africa shared their views on adopting computerised adaptive testing (CAT) prior to its implementation at their HEI in their respective modules, and their perceptions of such a testing methodology. Our study found that lecturers and students were comfortable engaging in online learning, with a large percentage being the most comfortable with assessing and completing exams, tests, and activities online. Positive perceptions of adopting CAT as an assessment tool for their qualifications were expressed, with the majority recommending their HEI to implement CAT.

Development of standard computerised adaptive test (CAT) settings for the EORTC CAT Core

AimsComputerised adaptive test (CAT) provides individualised patient reported outcome measurement while retaining direct comparability of scores across patients and studies. Optimal CAT measurement requires an appropriate CAT-setting, the set of criteria defining the CAT including start item, item selection criterion, and stop criterion. The European Organisation for Research and Treatment of Cancer (EORTC) CAT Core allows for assessing the 14 functional and symptom domains covered by the EORTC QLQ-C30 questionnaire. The aim was to present a general approach for selecting CAT-settings and to use this to develop a portfolio of standard settings for the EORTC CAT Core optimised for different purposes and populations.MethodsUsing simulations, the measurement properties of CATs of different length and precision were evaluated and compared allowing for identifying the most suitable settings. All CATs were initiated with the most informative QLQ-C30 item. For each domain two fixed-length and two fixed-precision standard CATs were selected focusing on efficiency (brief version) and precision (long), respectively.ResultsThe brief fixed-length CATs included 3–5 items each while the long versions included 5–8 items. The fixed-precision CATs aimed for reliability of 0.65–0.95 (brief versions) and 0.85–0.98 (long versions), respectively. Median sample size savings using the CATs compared to the QLQ-C30 scales ranged 20%-31%, although savings varied considerably across the domains.ConclusionThe EORTC CAT Core standard settings simplify selection of relevant and appropriate CATs. The CATs prioritise either brevity and efficiency or precision, but all provide increased measurement precision and hence, reduced sample size requirements compared to the QLQ-C30 scales. The CATs may be used as they are or modified to accommodate specific requirements.

Detecting uniform differential item functioning for continuous response computerized adaptive testing

Evaluating items for potential differential item functioning (DIF) is an essential step to ensuring measurement fairness. In this article, we focus on a specific scenario, namely, the continuous response, severely sparse, computerized adaptive testing (CAT). Continuous responses items are growingly used in performance-based tasks because they tend to generate more information than traditional dichotomous items. Severe sparsity arises when many items are automatically generated via machine learning algorithms. We propose two uniform DIF detection methods in this scenario. The first is a modified version of the CAT-SIBTEST, a non-parametric method that does not depend on any specific item response theory model assumptions. The second is a regularization method, a parametric, model-based approach. Simulation studies show that both methods are effective in correctly identifying items with uniform DIF. A real data analysis is provided in the end to illustrate the utility and potential caveats of the two methods.

Initial Validation of a Computerized Adaptive Test for Substance Use Disorder Identification in Adolescents

Purpose Computerized adaptive tests (CATs) are highly efficient assessment tools that couple low patient and clinician time burden with high diagnostic accuracy. A CAT for substance use disorders (CAT-SUD-E) has been validated in adult populations but has yet to be tested in adolescents. The purpose of this study was to perform initial evaluation of the K-CAT-SUD-E (i.e., Kiddy-CAT-SUD-E) in an adolescent sample compared to a gold-standard diagnostic interview. Methods Adolescents (N = 156; aged 11–17) with diverse substance use histories completed the K-CAT-SUD-E electronically and the substance related disorders portion of a clinician-conducted diagnostic interview (K-SADS) via tele-videoconferencing platform. The K-CAT-SUD-E assessed both current and lifetime overall SUD and substance-specific diagnoses for nine substance classes. Results Using the K-CAT-SUD-E continuous severity score and diagnoses to predict the presence of any K-SADS SUD diagnosis, the classification accuracy ranged from excellent for current SUD (AUC = 0.89, 95% CI = 0.81, 0.95) to outstanding (AUC = 0.93, 95% CI = 0.82, 0.97) for lifetime SUD. Regarding current substance-specific diagnoses, the classification accuracy was excellent for alcohol (AUC = 0.82), cannabis (AUC = 0.83) and nicotine/tobacco (AUC = 0.90). For lifetime substance-specific diagnoses, the classification accuracy ranged from excellent (e.g., opioids, AUC = 0.84) to outstanding (e.g., stimulants, AUC = 0.96). K-CAT-SUD-E median completion time was 4 min 22 s compared to 45 min for the K-SADS. Conclusions This study provides initial support for the K-CAT-SUD-E as a feasible accurate diagnostic tool for assessing SUDs in adolescents. Future studies should further validate the K-CAT-SUD-E in a larger sample of adolescents and examine its acceptability, feasibility, and scalability in youth-serving settings.

Comparison of screening methods for computer adaptive tests to predict reading and math performance

AbstractThe present study compared the diagnostic accuracy of a single computer adaptive test (CAT), Star Reading or Star Math, and a combination of the two in a gated screening framework to predict end‐of‐year proficiency in reading and math. Participants included 13,009 students in Grades 3–8 who had at least one fall screening score and end‐of‐year state test score in reading and math. First, diagnostic accuracy statistics were evaluated for a single screening measure to predict proficiency on end‐of‐year tests. Second, a gated screening framework was simulated to examine the diagnostic accuracy of a combination of screening measures (i.e., scores from the CATs and the end‐of‐year test). The diagnostic accuracy of each screening method was compared. Results suggest that diagnostic accuracy did not improve for the gated screening method when compared to the single screening method. The gated screening method tended to yield low sensitivity values (M = 0.42, range = 0.35–0.48) and high specificity values (M = 0.97, range = 0.95–0.99). The only condition to reach acceptable sensitivity and specificity (&gt;0.70) was a single reading screener predicting reading outcomes. Sample specific cut‐scores from receiver operating curve (ROC) analyses led to improved diagnostic accuracy outcomes relative to all other methods.

Reliability, validity, and simplification of the Chinese version of the Global Pain Scale in patients with rheumatoid arthritis

BackgroundPersistent pain is the most reported symptom in patients with rheumatoid arthritis (RA); however, effective and brief assessment tools are lacking. We validated the Chinese version of the Global Pain Scale (C-GPS) in Chinese patients with RA and proposed a short version of the C-GPS (s-C-GPS).MethodThe study was conducted using a face-to-face questionnaire survey with a multicenter cross-sectional design from March to December 2019. Patients aged > 18 years who met the RA diagnostic criteria were included. Based on the classical test theory (CTT) and the item response theory (IRT), we assessed the validity and reliability of the C-GPS and the adaptability of each item. An s-C-GPS was developed using IRT-based computerized adaptive testing (CAT) analytics.ResultsIn total, 580 patients with RA (mean age, 51.04 ± 24.65 years; mean BMI, 22.36 ± 4.07 kg/m2), including 513 (88.4%) women, were included. Most participants lived in a suburb (49.3%), were employed (72.2%) and married (91.2%), reported 9–12 years of education (66.9%), and had partial medical insurance (57.8%). Approximately 88.1% smoked and 84.5% drank alcohol. Analysis of the CTT demonstrated that all items in the C-GPS were positively correlated with the total scale score, and the factor loadings of all these items were > 0.870. A significant positive relationship was found between the Visual Analog Scale (VAS) and the C-GPS. IRT analysis showed that discrimination of the C-GPS was between 2.271 and 3.312, and items 6, 8, 13, 14, and 16 provided a large amount of information. Based on the CAT and clinical practice, six items covering four dimensions were included to form the s-C-GPS, all of which had very high discrimination. The s-C-GPS positively correlated with the VAS.ConclusionThe C-GPS has good reliability and validity and can be used to evaluate pain in RA patients from a Chinese cultural background. The s-C-GPS, which contains six items, has good criterion validity and may be suitable for pain assessment in busy clinical practice.Trial registrationThis cross-sectional study was registered in the Chinese Clinical Trial Registry (ChiCTR1800020343), granted on December 25, 2018.

New Dizziness Impact Measures of Positional, Functional, and Emotional Status Were Supported for Reliability, Validity, and Efficiency

ObjectiveTo calibrate the 25 items from the Dizziness Handicap Inventory (DHI) patient-reported outcome measure (PROM), using item response theory (IRT), into 1 or more item banks, and assess reliability, validity, and administration efficiency of scores derived from computerized adaptive test (CAT) or short form (SF) administration modes. DesignRetrospective cohort study. SettingOutpatient rehabilitation clinics. ParticipantsPatients (N=28,815; women=69%; mean age [SD]=60 [18]) included in a large national dataset and assessed for dizziness-related conditions who responded to all DHI items at intake. InterventionsNot applicable. Main Outcome MeasuresIRT model assumptions of unidimensionality, local item independence, item fit, and presence of differential item functioning (DIF) were evaluated. Generated scores were assessed for reliability, validity, and administration efficiency. ResultsPatients were treated in 976 clinics from 49 US states for either vestibular-, brain injury-, or neck-related impairments. Three unidimensional item banks were calibrated, creating 3 distinct PROMs for Dizziness Functional Status (DFS, 13 items), Dizziness Positional Status (DPS, 4 items), and Dizziness Emotional Status (DES, 6 items). Two items did not fit into any domain. A DFS-CAT and a DFS 7-item SF were developed. Except for 2 items by age groups and 1 item by main impairment, no items were flagged for DIF; DIF impact was negligible. Median reliability estimates were 0.91, 0.72, and 0.79 for the DFS, DPS, and DES, respectively. Scores discriminated between patient groups in clinically logical ways and had a large effect size (>0.8), with acceptable floor and ceiling effects (<15%), except for a floor effect for DPS (20.4%). DFS-CAT scores were generated using a median of 8 items; they correlated highly with full-bank scores (r=0.99). ConclusionThe 3 dizziness impact PROMs demonstrated moderate to high reliability, were valid, and highly responsive to change; thus, they are suitable for research and routine clinical administration.

Go Zika Go: feasibility study with modified motorized ride-on cars for the mobility of children with Congenital Zika Syndrome (CZS)

Purpose To explore the feasibility of an intervention using modified ride-on cars in terms of acceptability and potential changes in goal attainment, mobility, social function, and participation of children with Congenital Zika Syndrome (CZS). Materials and Methods A pre- and post-intervention feasibility study was conducted with children with CZS, levels IV and V of the Gross Motor Function Classification System (GMFCS). The intervention consisted of 12 weeks of training (3 times a week for 40 min per session) and 4 weeks of follow-up. The primary outcomes included adherence, satisfaction, and learning in mobility. Secondary outcomes encompassed goal attainment, mobility, social/cognitive function, and participation. Descriptive statistics were performed. To explore potential individual changes with the intervention, Wilcoxon test was used to analyze Pediatric Evaluation of Disability Inventory – Computer Adaptive Test (PEDI-CAT) data and Young Children’s Participation and Environment Measure (YC-PEM)/Participation and Environment Measure for Children and Youth (PEM-CY), along with standard error measurements of the PEDI-CAT domains Results Four children participated (median age 4.75 years; two females: three at level V on the GMFCS). Adherence was 75% of the total intervention time, and family members reported being satisfied or very satisfied. Children showed gains in learning the use of the modified ride-on cars and an increase in goal attainment after the intervention. Individual changes were observed in the PEDI-CAT domains (mobility and social/cognitive), but there were no significant changes in participation outcomes. Conclusions Children with CZS at GMFCS levels IV and V can learn to use motorized ride-on cars, attainment goals, and experience satisfaction.

The irtQ R package: a user-friendly tool for item response theory-based test data analysis and calibration.

Computerized adaptive testing (CAT) has become a widely adopted test design for high-stakes licensing and certification exams, particularly in the health professions in the United States, due to its ability to tailor test difficulty in real time, reducing testing time while providing precise ability estimates. A key component of CAT is item response theory (IRT), which facilitates the dynamic selection of items based on examinees' ability levels during a test. Accurate estimation of item and ability parameters is essential for successful CAT implementation, necessitating convenient and reliable software to ensure precise parameter estimation. This paper introduces the irtQ R package, which simplifies IRT-based analysis and item calibration under unidimensional IRT models. While it does not directly simulate CAT, it provides essential tools to support CAT development, including parameter estimation using marginal maximum likelihood estimation via the expectation-maximization algorithm, pretest item calibration through fixed item parameter calibration and fixed ability parameter calibration methods, and examinee ability estimation. The package also enables users to compute item and test characteristic curves and information functions necessary for evaluating the psychometric properties of a test. This paper illustrates the key features of the irtQ package through examples using simulated datasets, demonstrating its utility in IRT applications such as test data analysis and ability scoring. By providing a user-friendly environment for IRT analysis, irtQ significantly enhances the capacity for efficient adaptive testing research and operations. Finally, the paper highlights additional core functionalities of irtQ, emphasizing its broader applicability to the development and operation of IRT-based assessments.

Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record.

Patient-reported outcome (PRO) measures have become an essential component of quality measurement, quality improvement, and capturing the voice of the patient in clinical care. In 2004, the National Institutes of Health endorsed the importance of PROs by initiating the Patient-Reported Outcomes Measurement Information System (PROMIS), which leverages computer-adaptive tests (CATs) to reduce patient burden while maintaining measurement precision. Historically, PROMIS CATs have been used in a large number of research studies outside the electronic health record (EHR), but growing demand for clinical use of PROs requires creative information technology solutions for integration into the EHR. This paper describes the introduction of PROMIS CATs into the Epic Systems EHR at a large academic medical center using a tight integration; we describe the process of creating a secure, automatic connection between the application programming interface (API) which scores and selects CAT items and Epic. The overarching strategy was to make CATs appear indistinguishable from conventional measures to clinical users, patients, and the EHR software itself. We implemented CATs in Epic without compromising patient data security by creating custom middleware software within the organization's existing middleware framework. This software communicated between the Assessment Center API for item selection and scoring and Epic for item presentation and results. The middleware software seamlessly administered CATs alongside fixed-length, conventional PROs while maintaining the display characteristics and functions of other Epic measures, including automatic display of PROMIS scores in the patient's chart. Pilot implementation revealed differing workflows for clinicians using the software. The middleware software was adopted in 27 clinics across the hospital system. In the first 2 years of hospital-wide implementation, 793 providers collected 70,446 PROs from patients using this system. This project demonstrated the importance of regular communication across interdisciplinary teams in the design and development of clinical software. It also demonstrated that implementation relies on buy-in from clinical partners as they integrate new tools into their existing clinical workflow.

Adaptive Assessment of Visualization Literacy.

Visualization literacy is an essential skill for accurately interpreting data to inform critical decisions. Consequently, it is vital to understand the evolution of this ability and devise targeted interventions to enhance it, requiring concise and repeatable assessments of visualization literacy for individuals. However, current assessments, such as the Visualization Literacy Assessment Test (VLAT), are time-consuming due to their fixed, lengthy format. To address this limitation, we develop two streamlined computerized adaptive tests (CATs) for visualization literacy, A-VLAT and A-CALVI, which measure the same set of skills as their original versions in half the number of questions. Specifically, we (1) employ item response theory (IRT) and non-psychometric constraints to construct adaptive versions of the assessments, (2) finalize the configurations of adaptation through simulation, (3) refine the composition of test items of A-CALVI via a qualitative study, and (4) demonstrate the test-retest reliability (ICC: 0.98 and 0.98) and convergent validity (correlation: 0.81 and 0.66) of both CATs via four online studies. We discuss practical recommendations for using our CATs and opportunities for further customization to leverage the full potential of adaptive assessments. All supplemental materials are available at https://osf.io/a6258/.

Percepção de cuidadores de pré-escolares com TEA sobre seu comportamento e desempenho ocupacional durante a pandemia da COVID-19

Abstract Introduction Children with Autism Spectrum Disorder (ASD) face behavioral, sensory processing, cognitive, and language challenges that affect their ability to perform activities of daily living (ADL) and necessitate access to educational, therapeutic, and health services. The COVID-19 pandemic limited access to these essential services, exacerbating the challenges faced by this vulnerable population due to imposed social distancing measures. Objective To explore the perceptions of parent caregivers of preschoolers with ASD regarding their children’s behavior and performance in ADL during the COVID-19 pandemic. Method This cross-sectional, descriptive study utilized both quantitative and qualitative approaches, interviewing 60 caregivers of preschoolers with ASD. Participants were asked about any observed changes in their child’s behavior and ability to engage in ADL during the pandemic. Child characterization involved the Autism Classification System of Functioning: Social Communication (ACSF:SC), the Pediatric Evaluation of Disability Inventory - Computer Adaptive Test (PEDI-CAT), and the Sensory Profile 2 (SP-2). Interviews were documented, with 15 recorded and transcribed for thematic content analysis. Results Participants included 43 mothers, 15 couples, and two fathers, totaling 60 children, of whom 46 (76.7%) were boys aged between 3 and 5 years (mean age 53.6 ±10.18 months). Caregivers reported changes in behavior and performance in ADL, predominantly noting deterioration. Increases in stereotypic behaviors, rigidity, and hyperfocus were observed, along with decreased participation in ADL. While the presence of caregivers benefited some children, it also led to reported stress, particularly among mothers. Conclusion The study found a predominant trend of deteriorating behavior and performance in ADL among children with ASD during the pandemic, based on caregiver reports.

Improvements in Objective and Patient-Reported Measures of Physical Function Following an 8-Week Exercise Oncology Program

Background: Physical function is often impaired because of cancer and its treatment. Despite this, it is not common to comprehensively assess physical function in clinical practice. The purpose of this study was to examine the changes in objective and patient-reported physical function in adults with a history of cancer who completed an exercise program. Methods: The program was offered 2 d/wk for 8 weeks at an outpatient physical therapy clinic and consisted of resistance and aerobic exercises that were individualized to each participant. Objective physical function was assessed using the Bellarmine Norton Assessment Tool consisting of self-reported physical activity frequency, 2-minute step test, 30-second sit to stand, timed arm curl, Timed Up and Go, and scaled total score. Patient-reported physical function was assessed using the PROMIS-Physical Function (PF) Computerized Adaptive Test (CAT). Results: Significant differences (P &lt; .001) were found for all outcomes demonstrating improvements in objective and patient-reported physical function. Changes in raw and scaled total score for objective physical function displayed large Cohen d effect sizes (ES), physical activity (ES = 0.82), 2-minute step test (ES = 1.00), 30-second sit to stand (ES = 1.01), arm curl (ES = 1.01), Timed Up and Go (ES = 0.85), and scaled total score (ES = 1.31). Changes in PROMIS-PF CAT demonstrated a medium-large effect (ES = 0.78). Conclusion: The effectiveness of the exercise program was supported by significant improvements in objective and patient-reported physical function. The use of such outcomes provides a useful method to understand changes in physical function and supports the development of effective exercise methods for survivors of cancer.

Depressive Symptoms Associated With Social Participation in Older Adults Living With Mild Cognitive Impairment.

Mild Cognitive Impairment (MCI) affects 15% of adults 50 years old and over. Individuals living with MCI have shown decreased social participation, a critical activity as it may delay cognitive decline. Depression may be a key factor in limiting participation. This study is a secondary data analysis of 30 older adults living with MCI, looking for associations with participation. Participation was examined using the Patient Reported Outcomes Measurement Information System (PROMIS®) Satisfaction with Participation in Social Roles Computer Adaptive Test (CAT) form. Depressive symptoms were reported using the PROMIS Quality of Life in Neurological Disorders Depression CAT form and the Patient Health Questionnaire (PhQ-9) form. Results showed that demographics (age, sex) were not significantly associated with participation, but depressive symptoms were significantly associated. This suggests that adults living with MCI who have higher levels of depressive symptoms may be a uniquely vulnerable population who benefit from interventions that support participation.