Cut Score Research Articles

Relationships Between Functional Movement Quality and Sports Performance in Elite Figure Skating Athletes of China.

Zhitao, L, Junlong, D, Rui, Y, Leijiao, Y, Cheng, G, and Jun, Y. Relationships between functional movement quality and sports performance in elite figure skating athletes of China. J Strength Cond Res 38(11): e678-e685, 2024-This study aimed to determine the optimal functional movement screen (FMS) cut score for assessing the risk of sports injuries and to explore the correlations between functional movement quality and sports performance among elite figure skaters. A total of 22 elite figure pairs skaters in China underwent FMS, sports performance, and clinical examinations for sports injuries. The FMS cut score was determined using the receiver operating characteristic curve. Pearson's rank correlation analysis was used to quantify associations between the variables. (a) The optimal FMS cut score for assessing the risk of sports injuries was 13 (p = 0.008), with an area under curve of 0.838. The sensitivity and specificity were 0.889 and 0.692, respectively. (b) The trunk stability push-up (TSPU), an individual FMS test, predicted injuries in figure skaters (p = 0.015). (c) Moderate correlations were found between shoulder mobility and bench strength (r = -0.453, p = 0.034), counter movement jump (r = -0.582, p = 0.007), and 30-m sprint (r = 0.567, p = 0.009). A strong correlation was observed with squat jump (r = -0.774, p = 0.001). In addition, TSPU scores moderately correlated with 30-m sprint times (r = -0.511, p = 0.021) and 30-s average anaerobic power (r = 0.503, p = 0.024). These findings indicate that both the FMS total score ≤13 and the FMS individual score serve as indicators of injury risk in figure skaters. However, only the FMS individual score reflected the sports performance. This likely results from the similarity in kinetic chain systems and movement patterns involved in both the FMS individual test and sports performance, suggesting a promising assessment strategy for sports performance.

Evaluation of the Multidimensional Personality Questionnaire (MPQ) Unlikely Virtues Scale in the detection of underreporting.

The current investigation was designed to examine the impact of underreporting response bias on the Multidimensional Personality Questionnaire (MPQ) and the utility of the Unlikely Virtues scale in detecting such responding. Study participants were randomly assigned to either the underreporting (n = 100) or standard instruction (n = 224) conditions. All participants first completed a series of extratest measures under standard instruction prior to the MPQ administration. The results showed that individuals in the underreporting condition scored significantly higher on MPQ scales that reflect positive emotionality and self-control and lower on scales that indicate negative emotionality compared to those in the standard instruction. Moreover, the psychometric validity of MPQ scale scores against extratest measures was substantially attenuated in the underreporting condition compared to the standard instruction condition. The Unlikely Virtues scale differentiated between the conditions at a large effect size magnitude (Hedges' g = 1.21) and showed promising classification accuracy though lower cut scores than those reported in the MPQ manual performed substantially better with respect to sensitivity. Future research needs to replicate these findings in more naturalistic settings with other research designs. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Empirically Deriving Cut Scores in the Positive Behavioral Interventions and Supports (PBIS) Tiered Fidelity Inventory (TFI) Through a Bookmarking Process

We aimed to identify empirically valid cut scores on the positive behavioral interventions and supports (PBIS) Tiered Fidelity Inventory (TFI) through an expert panel process known as bookmarking. The TFI is a measurement tool to evaluate the fidelity of implementation of PBIS. In the bookmark method, experts reviewed all TFI items and item scores by difficulty in order estimated by an item response theory (IRT) model of TFI data. Over two sessions involving data review and discussion, panelists individually identified which corresponding cut scores in each TFI tier scale indicated minimally adequate fidelity in the TFI, leading to corresponding scores of 83.3% for TFI Tier 1, 84.6% for TFI Tier 2, and 82.4% for TFI Tier 3. Then, the proposed cut scores were compared with the original cut scores of 70% for percentage of schools meeting fidelity and the association between the cut scores and school discipline outcomes using an extant sample. The results of descriptive statistics and multiple t-tests suggested that the bookmark cut score in TFI Tier 1 was reachable and more strongly associated with improved school discipline outcomes. Findings for TFI Tier 2 and Tier 3 were less conclusive.

Extension and Further Replication of the Reliability, Criterion Validity, and Treatment Sensitivity of the PANSS10 and PANSS20 for Pediatric Trials.

Introduction: The Positive and Negative Syndrome Scale (PANSS) is a widely accepted outcome measure for pediatric schizophrenia trials; however, it has notable limitations. Psychometric investigations have shown a multifactorial structure and some items have limited utility assessing symptom severity in children. To address these issues, we developed and evaluated optimized 10- and 20-item PANSS short-forms (PANSS10 and PANSS20) using patient-level clinical trial data. This study further assesses these optimized forms using independent clinical trial data. Methods: We examined patient-level data from a randomized pediatric schizophrenia trial comparing paliperidone ER to aripiprazole. Data were accessed through the Yale Open Data Access (YODA) secure platform. Analyses included confirmatory factor analyses, graded response models, ω score reliability, internal consistency, sensitivity to change, and criterion validity versus the Clinical Global Impressions of Severity (CGI-S). Bland-Altman analyses examined score calibration versus the 30-item PANSS and inclusion cut scores. Results: Participants (N = 288) were ages 12 to 17 years (M = 15.3, SD = 1.46; 66% male). Total scores for the PANSS10 and PANSS20 showed strong correlations with the 30-item PANSS (0.90 and 0.97, respectively). Average inter-item correlations were 0.10 and 0.14 and ωTotal reliabilities were 0.74 and 0.85. Both PANSS10 and PANSS20 scores showed reliability >0.80-2.3 to 4.5 SD and -3.0 to 6.0 SD about mean symptom severity, respectively. Sensitivity to treatment was also similar (partial eta squared 0.23 and 0.22), as was correlation with CGI-S at baseline (0.45 and 0.48; not significantly different). The mean item-average discrepancy with the 30-item PANSS was 0.095 for PANSS10 and 0.033 for PANSS20. Conclusions: The optimized PANSS forms continue to show impressive reliability, validity, and calibration compared with the 30-item PANSS. Researchers should consider replacing the 30-item PANSS with the PANSS10 as a clinical outcome and screening measure due to its length and psychometric performance.

Minoritized language accreditation and equating. Some particularities

ABSTRACT Among the specificities associated with the accreditation of linguistic competence in minoritised languages, one of the most distinctive is the characterisation of the target population. In terms of diversity, test-takers come from the same geographical region, and most of them live in contact with the test language; in terms of quantity, there are fewer candidates than those assessed in L2 language around the world. These particularities have some important implications in any score-equating programme, and so on validity. In this paper, which focuses on Basque, we explore the conditions to define an equating design. We start by analysing the hypothesis of group equivalence among the editions of the HABE C1 test, and then compare some equating procedures. The research is carried out in two steps. The first (N = 14,981) analyses the sample compositions among test editions with relation to gender, age, place of birth, L1 and language of instruction; the second compares some equating procedures applied to the reading and listening test (N = 4765). The results support the hypothesis of group equivalence, and the equating procedures show total concordance around the cut scores. Some implications of the results are provided.

Performance replication of the Hospital Mental Health Risk Screen in ethnoracially diverse U.S. patients admitted through emergency care.

Patients admitted to hospitals after emergency care for injury or acute illness are at risk for later mental health problems. The American College of Surgeons Committee on Trauma Standards for care of injured patients call for mental health risk screening, and the Hospital Mental Health Risk Screen (HMHRS) accurately identified at-risk patients in a developmental study that included patients from five ethnoracial groups. Replication of these findings is essential, because initial positive results for predictive screens can fail to replicate if the items were strongly related to outcomes in the development sample but not in a new sample from the population the screen was intended for. Replication of the predictive performance of the 10-item HMHRS was studied prospectively in ethnoracially diverse patients admitted after emergency care for acute illness or injury in three hospitals across the U.S. Risk screen scores and follow-up mental health outcomes were obtained for 452 of 631 patients enrolled (72%). A cut score of 10 on the HMHRS correctly identified 79% of the patients who reported elevated levels of depression, anxiety, and PTSD symptoms two months post-admission (sensitivity) and 72% of the patients whose symptoms were not elevated (specificity). HMHRS scores also predicted well for patients with acute illness, for patients with injuries, and for patients who reported an Asian American/Pacific Islander, Black, Latinx, Multirace, or White identity. Predictive performance of the HMHRS was strong overall and within all five ethnoracial subgroups. Routine screening could reduce suffering and health care costs, increase health and mental health equity, and foster preventive care research and implementation. The performance of the HMHRS should be studied in other countries and in other populations of recent trauma survivors, such as survivors of disaster or mass violence.

Development of an Inconsistent Responding Scale for the Big Five Inventory-2

The Big Five Inventory − 2 (BFI-2) is a commonly used self-report assessment of normal personality trait domains (Extraversion, Agreeableness, Conscientiousness, Negative Emotionality, Open-Mindedness) and facets. To date, however, no direct measures of response distortion have been developed for it to identify potentially invalid responses. Such distortions (e.g., careless or random responding) can adversely impact data quality. The current study developed and provided initial validation data for an inconsistent responding scale within the BFI-2 to identify careless responders using two large undergraduate samples and a community sample. To create the scale, we first identified highly correlated BFI-2 item pairs in one undergraduate sample (N = 1,461) and then computed a total score by summing the absolute differences of these item pairs. This scale, the Detection of Response Inconsistency Procedure (DRIP), differentiated randomly generated and genuine data and generally correlated as expected with personality domains and other inconsistent responding scales across samples. The DRIP also incrementally predicted random data beyond a composite of items with exceptionally high or low base rates of endorsement from the Comprehensive Infrequency/Frequency Item Repository. We provide recommendations for DRIP cut scores that can detect careless responding while balancing sensitivity and specificity.

Morphological CBCT parameters for an accurate differentiation between nasopalatine duct cyst and the normal nasopalatine canal

BackgroundThe incisive foramen width was a traditional imaging criterion for diagnosing nasopalatine duct (NPD) cysts. Recent CBCT studies demonstrated significant dimensional variations of the nasopalatine canal, which raised questions about the accuracy of this criterion. This study investigated whether nasopalatine canal diameters assessed on CBCT images can accurately differentiate NPD cysts from normal nasopalatine canals.MethodsThe study included 19 patients with NPD cysts (12 (63.2%) males, 7 (36.8%) females, mean age 44.7 ± 13.3), and a control group of 164 patients (72 (43.9%) males, 92 (56.1%) females, mean age 47.25 ± 17.74). CBCT images were retrospectively evaluated. The following nasopalatine canal diameters were measured on reference sagittal, coronal, and axial reformation images: nasal opening anteroposterior (AP) and mediolateral (ML) diameter, oral opening AP (APOO) and ML (MLOO) diameter, nasopalatine canal length, minimum ML (minML) diameter, anterior wall expansion (AWE), nasopalatine canal angle, and the mid-level AP diameter (midAP). All parameters were compared between groups. Discriminant functional analysis (DFA) was applied to detect CBCT parameters that best differentiate the NPD cyst from the normal canal.ResultsPatients with NPD cyst had significantly greater values of APOO (7.06 ± 2.09 vs. 5.61 ± 1.70), MLOO (6.89 ± 2.95 vs. 3.48 ± 1.24), minML (2.88 ± 1.53 vs. 2.25 ± 1.09), AWE (2.15 ± 0.65 vs. 0.41 ± 0.67), and midAP (4.58 ± 1.61 vs. 2.48 ± 0.96). DFA showed MLOO, AWE, and midAP as the most accurate in distinguishing NPD cyst from the normal canal. When combined in the discriminant function equation X = 0.390·MLOO + 1.010·AWE + 0.288·midAP (cut score 1.669), the differentiation can be performed with a sensitivity and specificity of 98.8% and 76.9%, respectively.ConclusionNPD cysts can be accurately distinguished from the normal nasopalatine canal by measuring MLOO, AWE, and midAP diameter on CBCT images.

Using ROC Analysis to Refine Cut Scores Following a Standard Setting Process

In educational assessment, cut scores are often defined through standard setting by a group of subject matter experts. This study aims to investigate the impact of several factors on classification accuracy using the receiver operating characteristic (ROC) analysis to provide statistical and theoretical evidence when the cut score needs to be refined. Factors examined in the study include the sample distribution relative to the cut score, prevalence of the positive event, and cost ratio. Forty item responses were simulated for examinees of four sample distributions. In addition, the prevalence and cost ratio between false negatives and false positives were manipulated to examine their impacts on classification accuracy. The optimal cut score is identified using the Youden Index J. The results showed that the optimal cut score identified by the evaluation criterion tended to pull the cut score closer to the mode of the proficiency distribution. In addition, depending on the prevalence of the positive event and cost ratio, the optimal cut score shifts accordingly. With the item parameters used to simulate the data and the simulated sample distributions, it was found that when passing the exam is a low-prevalence event in the population, increasing the cut score operationally improves the classification; when passing the exam is a high-prevalence event, then cut score should be reduced to achieve optimality. As the cost ratio increases, the optimal cut score suggested by the evaluation criterion decreases. In three out of the four sample distributions examined in this study, increasing the cut score enhanced the classification, irrespective of the cost ratio when the prevalence in the population is 50%. This study provides statistical evidence when the cut score needs to be refined for policy reasons.

Design of a Tool Capable of Assessing Environmental Sociocultural Physical Factors Influencing Women's Decisions on When and Where to Toilet Within Real-World Settings: Protocol for the Build and Usability Testing of a Mobile App for Use by Community-Dwelling Women.

Although surveys and apps are available for women to report urination and bladder symptoms, they do not include their decisions regarding toileting. Real-world factors can interfere with toileting decisions, which may then influence bladder health. This premise lacks data per want of a robust data collection tool. The Prevention of Lower Urinary Tract Symptoms (PLUS) research consortium engaged a transdisciplinary team to build and test WhereIGo, a mobile data collection app for Android and iOS. The design goal was a comprehensive reporting system for capturing environmental, sociocultural, and physical factors that influence women's decisions for toileting. Aims include having (1) an innovative feature for reporting physiologic urge sensation when "thinking about my bladder" and shortly before "I just peed," (2) real-time reporting along with short look-back opportunities, and (3) ease of use anywhere. The development team included a plain language specialist, a usability specialist, creative designers, programming experts, and PLUS scientific content experts. Both real-time and ecological momentary assessments were used to comprehensively capture influences on toileting decisions including perceived access to toileting, degree of busyness or stress or focus, beverage intake amount, urge degree, or a leakage event. The restriction on the maximal number of taps for any screen was six. PLUS consortium investigators did pilot-testing. Formal usability testing relied on the recruitment of community-dwelling women at four PLUS research sites. Women used the app for 2 consecutive days. Outcome measures were the system usability scale (SUS; 0-100 range) and the functional Mobile Application Rating Scale (1-5 range). These scales were embedded at the end of the app. The estimated a priori sample size needed, considering the SUS cut point score set at ≥74, was 40 women completing the study. Funding was provided by the National Institute of Diabetes and Digestive and Kidney Diseases since July 2015. The integrity of the build process was documented through multiple 5-minute videos presented to PLUS Consortium and through WhereIGo screenshots of the final product. Participants included 44 women, with 41 (93%) completing data collection. Participants ranged in age from 21 to 85 years, were predominantly non-Hispanic White (n=25, 57%), college-educated (n=25, 57%), and with incomes below US $75,000 (n=27, 62%). The SUS score was 78.0 (SE 1.7), which was higher than 75% of the 500 products tested by the SUS developers. The mean functional Mobile Application Rating Scale score was 4.4 (SE 0.08). The build and informal acceptability testing were completed in 2019, enrollment for formal usability testing completed by June 2020, and analysis was completed in 2022. WhereIGo is a novel app with good usability for women to report toileting decisions, urination, and fluid intake. Future research using the app could test the influence of real-time factors on bladder health. RR1-10.2196/54046.

B - 131 Validation of the California Verbal Learning Test-II-Short Form Forced Choice Paradigm as an Indicator of Performance Validity

Abstract Objective Although the Forced Choice (FC) paradigm of the standard California Verbal Learning Test-Second Edition (CVLT-II) has been extensively validated as an embedded performance validity test (PVT), no studies have validated the CVLT-II Short Form (SF) against a standard battery of external PVTs. Method Participants included 98 veterans (Mage = 69.7 ± 9.0; 86.7% cisgender-male) who completed the CVLT-II-SF FC and the following independent criterion freestanding/embedded PVTS as part of comprehensive outpatient neuropsychological evaluations: the Advanced Clinical Solutions Word Choice Test, Test of Memory Malingering-Trial 1, and Reliable Digit Span from the Wechsler Adult Intelligence Test-Fourth Edition Digit Span subtest. Invalid performance was identified using two different criteria: (1) failing ≥2 PVTs in addition to other clinical indicators of poor engagement, or (2) failing one PVT. Receiver Operator Characteristic (ROC) curves tested the classification accuracy of identifying participants in the invalid group versus those with valid presentations and with or without cognitive impairment. Results Using both criteria, ROC analyses accurately identified invalidity (AUCs = 0.708–0.807) in the total sample, with cut-scores of ≤7 or ≤ 8 on the CVLT-II-SF FC exhibiting adequate specificities (0.900–0.986) and sensitivities of 0.407–0.500. Classification accuracy was acceptable for differentiating the invalid and cognitively intact groups (AUCs = 0.734–0.829) at a cut score of ≤8 (specificities = 0.923–0.958, sensitivities = 0.500–0.704), but was variably acceptable for differentiating the invalid and cognitively impaired groups (AUCs = 0.684–0.794; cut-score of ≤7 with specificities = 0.962–0.978 and sensitivities = 0.229–0.407). Conclusions Our findings support the CVLT-II-SF FC trial as an embedded PVT, particularly when differentiating cognitively intact examinees from those with noncredible presentations.

Psychometric validation of the Italian Self-Report Symptoms inventory (SRSI): Factor structure, construct validity, and diagnostic accuracy

The Self-Report Symptom Inventory (SRSI) is a novel tool designed to detect symptom overreporting and other forms of noncredible responding. Unlike existing scales, the SRSI includes genuine and pseudosymptoms scales covering cognitive, affective, motor, pain, and post-traumatic stress disorder domains. The present study aims to investigate the psychometric properties of the Italian Version of the SRSI (SRSI-It), in particular, its factor structure, reliability, convergent and discriminant validity, and diagnostic accuracy. Data from 1180 healthy participants showed a hierarchical structure with higher-order constructs for genuine symptoms and pseudosymptoms, each comprising five subscales. The SRSI-It showed a strong convergent validity with the Structured Inventory of Malingered Symptomatology and discriminant validity through low correlations with the Psychopathic Personality Inventory-Revised. Receiver operating characteristic analysis determined cut scores of 6 (95% specificity) and 9 (98% specificity) for pseudosymptoms, with a Ratio Index score of 0.289 (82% specificity). In summary, the SRSI-It appears to be a promising tool for identifying symptom exaggeration in clinical and forensic contexts, ultimately enhancing the quality and reliability of evaluations in these contexts.

Digital Clock and Recall: a digital, process-driven evolution of the Mini-Cog.

Alzheimer's disease and related dementias (ADRD) represent a substantial global public health challenge with multifaceted impacts on individuals, families, and healthcare systems. Brief cognitive screening tools such as the Mini-Cog© can help improve recognition of ADRD in clinical practice, but widespread adoption continues to lag. We compared the Digital Clock and Recall (DCR), a next-generation process-driven adaptation of the Mini-Cog, with the original paper-and-pencil version in a well-characterized clinical trial sample. DCR was administered to 828 participants in the Bio-Hermes-001 clinical trial (age median ± SD = 72 ± 6.7, IQR = 11; 58% female) independently classified as cognitively unimpaired (n = 364) or as having mild cognitive impairment (MCI, n = 274) or dementia likely due to AD (DLAD, n = 190). MCI and DLAD cohorts were combined into a single impaired group for analysis. Two experienced neuropsychologists rated verbal recall accuracy and digitally drawn clocks using the original Mini-Cog scoring rules. Inter-rater reliability of Mini-Cog scores was computed for a subset of the data (n = 508) and concordance between Mini-Cog rule-based and DCR scoring was calculated. Inter-rater reliability of Mini-Cog scoring was good to excellent, but Rater 2's scores were significantly higher than Rater 1's due to variation in clock scores (p < 0.0001). Mini-Cog and DCR scores were significantly correlated (τ B = 0.71, p < 0.0001). However, using a Mini-Cog cut score of 4, the DCR identified more cases of cognitive impairment (n = 47; χ 2 = 13.26, p < 0.0005) and Mini-Cog missed significantly more cases of cognitive impairment (n = 87). In addition, the DCR correctly classified significantly more cognitively impaired cases missed by the Mini-Cog (n = 44) than vice versa (n = 4; χ 2 = 21.69, p < 0.0001). Our findings demonstrate higher sensitivity of the DCR, an automated, process-driven, and process-based digital adaptation of the Mini-Cog. Digital metrics capture clock drawing dynamics and increase detection of diagnosed cognitive impairment in a clinical trial cohort of older individuals.

Family Caregivers of Persons with Bipolar Disorder: Caregiver Demographics and Need and Preference for Intervention

The unpredictability of bipolar disorder is highly distressing for family caregivers, who differ in their needs and preferences for stress-reducing or educational interventions. Applying Ryan and Sawin’s model, this study examined associations between caregiver demographics (age, gender, and race) as contextual factors and caregiver needs and preferences for three interventions as process factors within a preliminary descriptive analysis of 306 family caregivers from a randomized clinical trial. Caregiver needs for education about bipolar disorder, biofeedback, and Resourcefulness Training© were determined by established cut scores on validated measures of bipolar knowledge, heart rate variability, and resourcefulness. Frequencies for need and preference for intervention were compared by caregiver age, gender, and race. Discrepancies between caregiver need and preference for interventions were analyzed. Non-White caregivers showed greater need for education (X 2=33.68, p < 0.001). Middle-aged caregivers showed greatest need for biofeedback (X 2=19.58, p < 0.001). Need for Resourcefulness Training© was similar across age, gender, and race. We found 58% needed biofeedback, 34% education, and 18% Resourcefulness Training©; 46% of those in the preference group chose Resourcefulness Training©. Further caregiver intervention research should consider the effect of caregiver needs and preferences on their health. The findings support the essentiality of assessing caregiver demographics, needs, and preferences before implementing interventions.

Assessing the utility of the PC-PTSD-5 as a screening tool among a cancer survivor sample.

Hematopoietic stem cell transplantation (HCT) is an intensive and invasive procedure used in cancer treatment that can lead to posttraumatic stress disorder (PTSD) symptoms. These symptoms are frequently overlooked in oncology and general health care settings. The suitability and utility of the Primary Care PTSD Screen for DSM-5 (PC-PTSD-5) within the cancer population remains uncertain. This study aims to evaluate its performance as a brief (five-item) case-finding screening alternative to the longer (20-item) PTSD Checklist for DSM-5 (PCL-5) in survivors who received an HCT 1 to 5 years ago. A total of 817 cancer survivors completed the PC-PTSD-5 and PCL-5 during recruitment for a randomized clinical trial. Optimal cut scores for identifying probable PTSD and item performance were determined using indices correcting for chance and item response theory analyses. Of the HCT sample, 10.4% screened as positive for probable DSM-5 PTSD using the PCL-5. The PC-PTSD-5 exhibited strong internal consistency and significant associations with PCL-5 scores (total, r=.82; items, rs=.56-.61). A cutoff score of 2 provided optimal sensitivity for screening (κ[Se]=.95), whereas a cut score of 4 demonstrated the highest efficiency for detecting a probable DSM-5 PTSD diagnosis on the PCL-5 (κ[Eff]=.39). Item response theory analyses indicated that item 4 (numbing) of the PC-PTSD-5 yielded the most informative data, with other items potentially lacking incremental utility. Although not an instrument validation study, these findings offer efficient evidence for using the PC-PTSD-5 as a succinct screening tool among cancer survivors in a clinical context. ClinicalTrials.gov, NCT04058795, registered 8/16/2019.

Evaluation of Fall Risk in Hemodialysis Patients within the Scope of Quality Improvements: A Comparison of Two Assessment Scales

Introduction: Patient falls are one of the most important quality indicators in healthcare. Aim: It was aimed to compare two tools used to evaluate fall risk in hemodialysis patients. Method: The mean age of the patients was 58.72 ± 14.49 years and 62% were male. The mean duration of treatment was 4.98 ± 4.71 years. Within the last year, 33% of the patients had a history of falling. The patients' fall risk was assessed using the Itaki Fall Risk Scale and Dialysis Fall Risk Index. The relationship between patients' age, gender, duration of hemodialysis treatment, and the status of having a chronic disease and falls were examined. Results: According to the Itaki Scale, 57% were in the ''high-risk'' group. According to the Dialysis Fall Risk Index, 64% were in the "very high risk" group. The mean Itaki Fall Risk Scale score was 4.75 ± 3.92, and the mean of the Dialysis Fall Risk Index was 7.59 ± 1.92. According to the cut-of score, the sensitivity and specifiity of the Itaki Fall Risk Scale were 97% and 99.6%, and the Dialysis Fall Risk Index was 93.9% and 99.6%. The positive and negative predictive value were 56.1% and 97.7%, respectively for the Itaki Fall Risk Scale. These points were 51.6% and 100% for the Dialysis Fall Risk Index. Conclusion: Both scales were effctive in improving care quality in hemodialysis centers. Itaki Scale may be preferred due to its ease of use and patient compliance.

Multivariable utility of the Memory Integrated Language and Making Change Test

Recent reports indicate that the Memory Integrated Language Test (MIL) and Making Change Test Abbreviated Index (MCT-AI), two web-based performance validity tests (PVTs), have good sensitivity and specificity when used independently. This study investigated whether using these PVTs together could improve the detection of invalid performance in a mixed neuropsychiatric sample. Participants were 129 adult outpatients who underwent a neuropsychological evaluation and were classified into valid (n = 104) or invalid (n = 25) performance groups based on several commonly used PVTs. Using cut scores of ≤41 on the MIL and ≥1.05 on the MCT-AI together enhanced classification accuracy, yielding an area under the curve of .84 (95% CI: .75, .93). As compared to using the MIL and MCT-AI independently, the combined use increased the sensitivity from .10-.31 to.70 while maintaining ≥.90 specificity. Findings also indicated that failing either the MIL or MCT-AI was associated with somewhat lower cognitive test scores, but failing both was associated with markedly lower scores. Overall, using the MIL and MCT-AI together may be an effective way to identify invalid test performance during a neuropsychological evaluation. Furthermore, pairing these tests is consistent with current practice guidelines to include multiple PVTs in a neuropsychological test battery.

Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study.

This study aimed to compare and evaluate the efficiency and accuracy of computerized adaptive testing (CAT) under 2 stopping rules (standard error of measurement [SEM]=0.3 and 0.25) using both real and simulated data in medical examinations in Korea. This study employed post-hoc simulation and real data analysis to explore the optimal stopping rule for CAT in medical examinations. The real data were obtained from the responses of 3rd-year medical students during examinations in 2020 at Hallym University College of Medicine. Simulated data were generated using estimated parameters from a real item bank in R. Outcome variables included the number of examinees’ passing or failing with SEM values of 0.25 and 0.30, the number of items administered, and the correlation. The consistency of real CAT result was evaluated by examining consistency of pass or fail based on a cut score of 0.0. The efficiency of all CAT designs was assessed by comparing the average number of items administered under both stopping rules. Both SEM 0.25 and SEM 0.30 provided a good balance between accuracy and efficiency in CAT. The real data showed minimal differences in pass/ fail outcomes between the 2 SEM conditions, with a high correlation (r=0.99) between ability estimates. The simulation results confirmed these findings, indicating similar average item numbers between real and simulated data. The findings suggest that both SEM 0.25 and 0.30 are effective termination criteria in the context of the Rasch model, balancing accuracy and efficiency in CAT.

Standard setting OSCE: A comparison of arbitrary and Hofstee methods in a low stake OSCE

Objectives: To compare the cut scores and pass/fail rates achieved by arbitrary 60% method and Hofstee method in an undergraduate year 4 end semester objective structured clinical examination (OSCE) and check the possibility of using Hofstee method of standard setting in future exams. Method: 102 medical students of year 4 underwent a 10 station OSCE exam conducted in a state of art simulation lab in 3 cycles. The cut scores were calculated using arbitrary method aiming at 60% of task achievement and by Hofstee method. The student’s obtained scores were compared for cut scores and pass rates for individual stations and the entire exam. Results: The arbitrary and Hofstee methods of standard setting leads to different cut scores. For the individual stations it was 60% vs 65-70% and for the overall score it was 60% vs 70%. The percentage of students failing the exam is 13.7% based on arbitrary scores and is 29.4% when Hofstee cut score is applied. Conclusions: The two methods lead to different cut scores and students’ failure rates. Overall, Hofstee method is more appropriate for assessing competencies in an OSCE exam in medical schools as it leads to calculation of cut scores based on the difficulty level of the station/exam and the examiners expected level of performance by the students. Keywords: Objective Structured Clinical Examination, Standard Setting, Hofstee Method, Arbitrary Method

Social challenges, autism spectrum disorder, and attention deficit/hyperactivity disorder in youth with neurofibromatosis type I

Objective Youth with neurofibromatosis type I (NF1) demonstrate high rates of Autism Spectrum Disorder (ASD) and Attention Deficit/Hyperactivity Disorder (ADHD), which often have overlapping behaviors. Diagnostic clarity is important to guide services. This study evaluated ASD classification in NF1 using various methods and whether those with ADHD suspicion have more social challenges associated with ASD. Method 34 youth with NF1 (M age = 10.5 ± 1.6 years), completed ASD assessments that combined direct observation and informant ratings to yield a Clinician Best Estimate (CBE) classification. Caregivers rated ASD-related social challenges using the Social Responsiveness Scale- 2nd Edition (SRS-2). Results ASD classification varied depending on the method, ranging from 32% using low-threshold SRS-2 cut-scores (T ≥ 60) to under 6% when combining cut scores for diagnostic observational tools and stringent SRS-2 cut-scores (T ≥ 70). 14.7% had a CBE ASD classification. 44% were judged to have autism traits associated with a non-ASD diagnosis. The 52.9% with a suspicion of ADHD had higher SRS-2 scores than those without ADHD, F (7, 26) = 3.45, p < .05, Wilk’s lambda = 0.518, partial eta squared = 0.482. Conclusions Findings highlight the importance of rigorous diagnostic methodology when evaluating ASD in NF1 to inform the selection of targeted interventions for socialization challenges in NF1.