Inter-rater Research Articles

57 Agreement in visual quality assessment of donated livers and kidneys among consultant surgeons

Abstract Introduction Surgeons' visual assessments represent the most common reason for non-utilisation of kidneys and livers retrieved for transplantation. While variability in organ acceptance between transplant centres is recognised, the influence of differences in subjective visual evaluations remains unclear. This study investigates inter-rater agreement amongst consultant transplant surgeons visually assessing organs for transplantation. Methods The study included 433 photographs (293 liver, 140 kidney), assessed by at least two transplant surgeons with over five years’ experience. Participants included eight surgeons from five centres for kidney photographs, and ten surgeons from six centres for liver photographs. Inter- and intra-rater agreements were determined with Gwet’s agreement coefficient (AC). Results There was excellent inter-rater agreement for liver steatosis across all levels (none/mild/moderate/severe), AC 0.81 (95%CI:0.76-0.86). Overall quality (good/moderate/poor) agreement was significantly lower in “good" vs “poor” quality livers, AC 0.86 (95%CI:0.75-0.97) vs AC 0.57 (95%CI:0.43-0.71), p<0.05. For kidney assessments, inter-rater agreement on global perfusion was excellent across all levels (good/fair/poor/patchy), AC 0.83 (95%CI:0.73-0.93). Agreement on overall quality was mixed, ranging from AC 0.94 (95%CI:0.73-1.00) for “good” kidneys, to AC 0.70 (95%CI:0.56-0.84) for “moderate” kidneys. Intra-rater agreement for both organs was consistently high (AC>0.86). Conclusions Despite excellent agreement among consultants on specific visual characteristics, there was a notable decrease in agreement when assessing overall quality. Clinicians concur on objective findings, but differ in how they weigh these observations when determining an organ's suitability for transplantation. These subjective differences may contribute to the variable organ acceptance rates across centres.

Deep Learning-Based Contrast Boosting in Low-Contrast Media Pre-TAVR CT Imaging.

Deep Learning-Based Contrast Boosting in Low-Contrast Media Pre-TAVR CT Imaging.

Which Measuring Method Is Better for Reflecting Subtalar Joint Stiffness?

Background/Objectives: A standard method for the measurement of subtalar joint motion has yet to be developed. This study aimed to determine which of the two methods (AMA and modified McBride) for the measurement of subtalar joint motion better reflects subtalar joint stiffness. Methods: We evaluated the intraobserver and interobserver reliability and validity of the two methods in the normal group (n = 50) and the patient group (n = 90; 30 with normal hindfoot, 30 with ankle joint pathology, and 30 with subtalar joint pathology). We assessed the intraclass correlation coefficient (ICC) and receiver operating characteristic (ROC) curve for statistical analysis. Results: In the normal group, the intraobserver reproducibility and interobserver reliability of the AMA method were better than those of the modified McBride method. In the hindfoot patient group, both methods showed excellent interobserver reliability (ICC > 0.75). The results of the ROC curve also indicated that the modified McBride method (AUC: 0.919, 0.813) was better than the AMA method (AUC: 0.715, 0.749) in reflecting subtalar joint stiffness. Conclusions: In patients with subtalar joint lesions, the modified McBride method had a relatively better validity and was more appropriate than the AMA method for measuring the stiffness of the subtalar joint.

Comparing Short Cognitive Screening Instruments in an Outreach Memory Clinic in Primary Care

Few studies have investigated the possibility of offering outreach from hospital-based memory clinic services to primary care. Such models could potentially improve access to specialised mental healthcare. We report on the reliability and validity of a pilot cognitive screening pathway in general practice (GP) in Ireland. Consecutive patients with memory complaints attending two primary care clinics were screened and diagnosed clinically by a physician-run memory clinic in GP. Follow-up in secondary care confirmed the diagnosis. Inter-rater reliability (IRR) and diagnostic validity of the Quick Mild Cognitive Impairment (Qmci) screen was compared to the Montreal Cognitive Assessment (MoCA) and General Practitioner Assessment of Cognition (GPCOG). In all, 63 patients, 31 with subjective memory complaints (SMC), 16 with mild cognitive impairment and 16 with dementia were screened. Their median age was 73 and 67% were female. The IRR of the Qmci screen between GP and clinic was excellent (r = 0.89). The Qmci was more accurate than the GPCOG in identifying cognitive impairment; the area under the curve (AUC) was 0.95 versus 0.80 (p = 0.008). The Qmci and MoCA had similar accuracy, with an AUC of 0.95 versus 0.91 (p = 0.117), respectively, but was significantly shorter (p < 0.001), suggesting it may be a useful instrument in this setting. Based on these results, a definitive study is now planned to examine the benefits and challenges of utilizing these instruments as part of establishing an outreach memory clinic service in primary care.

Collaborative human-AI risk annotation: co-annotating online incivility with CHAIRA

Introduction. Collaborative human-AI annotation is a promising approach for various tasks with large-scale and complex data. Tools and methods to support effective human-AI collaboration for data annotation are an important direction for research. In this paper, we present CHAIRA: a Collaborative Human-AI Risk Annotation tool that enables human and AI agents to collaboratively annotate online incivility. Method. We leveraged Large Language Models (LLMs) to facilitate the interaction between human and AI annotators and examine four different prompting strategies. The developed CHAIRA system combines multiple prompting approaches with human-AI collaboration for online incivility data annotation. Analysis. We evaluated CHAIRA on 457 user comments with ground truth labels based on the inter-rater agreement between human and AI coders. Results. We found that the most collaborative prompt supported a high level of agreement between a human agent and AI, comparable to that of two human coders. While the AI missed some implicit incivility that human coders easily identified, it also spotted politically nuanced incivility that human coders overlooked. Conclusions. Our study reveals the benefits and challenges of using AI agents for incivility annotation and provides design implications and best practices for human-AI collaboration in subjective data annotation.

Reliability and validity of the Chinese version of the stroke upper limb capacity scale

Purpose To evaluate the reliability and validity of the Chinese version of the Stroke Upper Limb Capacity Scale (SULCS) compared to the Wolf Motor Function Test (WMFT) and the Fugl-Meyer Assessment-Upper Extremity (FMA-UE). Materials and methods Patients with hemiplegia after stroke (n = 60). The intraclass correlation coefficient (ICC) and corresponding 95% confidence intervals (CI) were used to explore the reliability, and Spearman’s correlation coefficient was used to analyze validity. Floor and ceiling effects were counted. Results The ICCs for inter-rater reliability and intra-rater reliability were ICC = 0.985 (95%CI = 0.966–0.992) and ICC = 0.982 (95%CI = 0.970–0.989), respectively. The SULCS demonstrated a strong correlation with the WMFT (r = 0.922, p < 0.001) and FMA-UE (r = 0.921, p < 0.001) at baseline. The WMFT and FMA at week 3 were as strong as those at baseline (r = 0.946, p < 0.001 and r = 0.952, p < 0.001, respectively). There was a strong longitudinal correlation between the changes in SULCS and WMFT and FMA-UE from baseline to week 3 (r = 0.745, p < 0.001 and r = 0.768, p < 0.001, respectively). No floor and ceiling effects were detected in any assessment. Conclusions Our analysis of the psychometric properties supports the use of the Chinese version of the SULCS to measure upper limb capacity in patients with hemiplegia after stroke.

Evaluating ChatGPT's Reliability in Second Language Acquisition (SLA): Insights on Language Skills and Technology's Role

Artificial Intelligence (AI) has become a transformative tool across various fields, including education, offering innovative opportunities for second language (L2) learning and teaching. The potential of AI-driven tools like ChatGPT in Second Language Acquisition (SLA) remains an area of emerging interest, with implications for enhancing pedagogical approaches and providing scalable, interactive support for learners and educators. This study evaluates the reliability and quality of ChatGPT-4.0’s responses to 48 key questions related to SLA, encompassing themes such as technology's role and language skills (reading, writing, listening, and speaking). The questions were rated by five expert linguists using a Likert scale to assess accuracy, and relevance. Statistical analyses, including the Kruskal-Wallis H test (p &lt; 0.05) and post-hoc pairwise comparisons, revealed significant differences among evaluators (p &lt; 0.001), with most responses rated as good (26.25%) or very good (55.41%). Despite these positive outcomes, Fleiss’s Kappa coefficient (0.006) highlighted low inter-rater agreement, signaling a need for standardized evaluation frameworks. While ChatGPT shows promise in addressing diverse SLA topics, its limitations—such as occasional inaccuracies—underscore the necessity for ongoing research to optimize its utility in educational contexts. This study contributes to SLA pedagogy by demonstrating the potential and boundaries of integrating AI into language learning, advocating for cautious and evidence-based adoption.

Can MRI be apotential substitute for CT in cephalometric analysis for radiation-free diagnoses?

The primary objective of this study was to investigate the feasibility of magnetic resonance imaging (MRI) usage over computed tomography (CT) to perform three-dimensional (3D) cephalometric analyses. The secondary objective is to find intra- and interobserver reliability of manual cephalometric landmarks identification in both CT and MRI scan data. Data from 40patients were used in this study, with orthodontists manually identifying 37landmarks on both CT and MRI scans. The interclass correlation coefficient (ICC) was calculated individually for both CT and MRI scan data to find intra- and interobserver reliability. In addition to ICC, paired t‑test and mean error were also calculated. Ground truth landmarks were calculated by considering the mean values of manually located 37landmarks by observers for both CT and MRI. Thirty-seven cephalometric measurements (29linear, 6angular, and 2ratios) were measured using 37ground truth landmarks. Mean error (ME) between CT and MRI measurements was calculated and paired t‑test was performed to find the reliability of MRI usage over CT. Bland-Altman analysis was also performed on the measurements to check the agreement between CT and MRI. The intra- and interobserver reliability was found to be reliable (ICC > 0.98, and P > 0.05) for all 37landmarks in both CT and MRI. The ME for linear measurements was found to be 1.81 mm for hard tissue, 1.72 mm for soft tissue, and 1.53° for hard tissue angular measurements between CT and MRI. The paired t‑test performed on measurements between CT and MRI proved to be statistically insignificant (p > 0.05). The Bland-Altman analysis also showed strong agreement and low systemic bias between CT and MRI data. The strong ICC and Pvalues shows the high reliability and reproducibility of manual landmark identification on both CT and MRI. The ME for the linear and angular measurements between CT and MRI was found to be well within acceptable limits. The results of paired t‑test and Bland-Altman analyses for cephalometric measurements between CT and MRI has shown strong evidence supporting the use of MRI as asubstitute for CT.

Comparison of Smartphone Applications with Traditional Tools in the Assessment of Patellofemoral Pain: Validity and Reliability Study

Objectives: Patellofemoral pain (PFP) is one of the most confusing and clinically challenging chronic diseases. The aim of this study was to determine the validity and reliability of the smartphone goniometer application in patients with PFP by comparing it with a universal and digital inclinometer. Materials and Methods: Twenty-seven patients with PFP were included in this study. Flexion range of motion for the knee joint and flexion/extension, abduction, and internal/external rotation range of motion for the hip joint were measured by two examiners using a smartphone application, a digital inclinometer, and a universal goniometer. To assess inter-rater reliability, the two measurements made by the first observer were evaluated at 24-48 hour intervals. To assess intra-rater reliability, the measurements of both observers were compared. Results: With all three methods, active knee and hip range of motion measurements in PSS patients showed high intra-rater and inter-rater reliability (ICC = 0.69-0.97). Concurrent validity analysis also showed statistically significant, moderate to strong correlations between the three methods (r = 0.562-0.993). SEM and MDC were highest in the goniometer measurement and were intra-observer (3.77-7.94° and 10.44-21.99°, respectively) and inter-observer (2.61-9.45° and 7.23-27.54°, respectively). Conclusion: The smartphone app, inclinometer, and universal goniometer are valid and reliable for measuring lower limb ROM in PFA patients. They can be used in the clinic.

Radiographic features of Wu et al. type A2 congenital thumb duplication and implications for management: new subtypes and surgical strategies

ObjectiveThis study aimed to assess the radiographic features of patients diagnosed with congenital thumb duplication (CTD) type A2 based on the Wu et al. classification, describe the different subtypes of duplications and propose a classification system that permits identifying various surgical strategies.MethodsWe evaluated 665 patients (680 thumbs) diagnosed with type A2 CTDs by examining the alignment of the interphalangeal (IP) and metacarpophalangeal (MP) joints of the primary thumb on posteroanterior (PA) radiographs. The classification system has four types: Type I (no deviation); Type II (ulnar deviation); Type III (hypertrophic epiphysis); and Type IV (convergent). Types I-IV were compared to Hung et al.'s system Type A-D (Hypoplastic, Ulnar Deviation, Divergent, and Convergent).ResultsOf the 680 fingers, 436 (64.1%) were determined to be Wassel type IV while 244 (35.9%) were classified as Wassel type VII. All of the 436 fingers could be categorized according to the subtypes of the Hung et al. system; in particular, 369 (84.6%) were identified as type B, 52 (11.9%) as type D, and 15 cases (3.4%) as type C. The proposed classification system worked effectively for all CTDs (n = 680). 494 cases were classified as type II (72.6%), while 75 cases were classified as type I (11.0%). The remaining 111 cases were further classified as either type IV (9.3%) or type III (7.1%). The Wu et al. systems showed excellent intra-rater (0.881) and inter-rater (0.873) reliability compared to the Hung et al. systems (0.842 and 0.823, respectively).ConclusionsThe proposed radiographic pathoanatomical system has the potential to improve communication and guide optimal procedure selection for different subtypes of CTD depending on the attachment of the extra digit to the main thumb and the alignment of the interphalangeal and metacarpophalangeal joints of the primary thumb (Wu et al. type A2).Level of evidenceIII

High identification and positive-negative discrimination but limited detailed grading accuracy of ChatGPT-4o in knee osteoarthritis radiographs.

To explore the potential of ChatGPT-4o in analysing radiographic images of knee osteoarthritis (OA) and to assess its grading accuracy, feature identification and reliability, thereby helping surgeons to improve diagnostic accuracy and efficiency. A total of 117 anterior‒posterior knee radiographs from patients (23.1% men, 76.9% women, mean age 69.7 ± 7.99 years) were analysed. Two senior orthopaedic surgeons and ChatGPT-4o independently graded images with the Kellgren-Lawrence (K-L), Ahlbäck and International Knee Documentation Committee (IKDC) systems. A consensus reference standard was established by a third radiologist. ChatGPT-4o's performance metrics (accuracy, precision, recall and F1 score) were calculated, and its reliability was assessed via two evaluations separated by a 2-week interval, with intraclass correlation coefficients (ICCs) determined. ChatGPT-4o achieved a 100% identification rate for knee radiographs and demonstrated strong binary classification performance (precision: 0.95, recall: 0.83, F score: 0.88). However, its detailed grading accuracy (35%) was substantially lower than that of surgeons (89.6%). Severe underestimation of OA severity occurred in 49.3% of the cases. Interrater reliability for surgeons was excellent (ICC: 0.78-0.91), whereas ChatGPT-4o showed poor initial consistency (ICC: 0.16-0.28), improving marginally in the second evaluation (ICC: 0.22-0.39). ChatGPT-4o has the potential to rapidly identify and binary classify knee OA on radiographs. However, its detailed grading accuracy remains suboptimal, with a notable tendency to underestimate severe cases. This limits its current clinical utility for precise staging. Future research should focus on optimising its grading performance and improving accuracy to enhance diagnostic reliability. Level III retrospective comparative study.

Interrater reliability of the SPARK36, a novel instrument to guide preventive nurse-led consultations with parents of three-year olds.

Interrater reliability of the SPARK36, a novel instrument to guide preventive nurse-led consultations with parents of three-year olds.

Remote Assessment of Pediatric Anthropometrics.

To assess the reliability and fidelity of caregiver- and examiner-administered anthropometric measurements in English- and Spanish-speaking children aged 0-17years. Participants (N = 199; median age, 4.69years; interquartile range, 1.58-10.37; 46.7% male) were recruited in 6 sites across the United States in May-August 2023. Examiners were trained by a pediatric nurse; caregivers received well-developed administration materials. The key anthropometric outcomes assessed were infant length/child height, weight, percent body fat (PBF), head size, and waist circumference. Concordance between caregiver/examiner measurements was assessed using interrater reliability estimates, technical error of measurement (TEM), and mean absolute difference (MAD). Secondary outcomes included reported ease of use and language used. High concordance was observed between caregivers and examiners, with reliability coefficients ranging from 0.91 to >0.99 and intraclass correlation coefficient values from 0.92 to >0.99 across all assessments and age groups. The TEM was low (height = 2.10cm, weight = 0.39kg, PBF = 2.33%; head circumference = 0.73cm, waist circumference = 3.65cm) as were MAD estimates (height = 2.12cm, 95% confidence interval [CI], 1.82-2.42; weight = 0.35kg, 95% CI, 0.29-0.41; PBF = 2.39%, 95% CI, 1.89-2.88; head size = 0.71cm, 95% CI, 0.54-0.89; waist circumference = 3.15cm, 95% CI, 2.41-3.88). Across multiple metrics, there was high concordance between caregiver- and examiner-administered measurements, providing evidence of interchangeability between remote and in-person assessments of anthropometric outcomes. Remote anthropometric assessment appears feasible and low burden and has comparably high validity and reliability to in-person methods.

Assessing Infant Gross Motor Performance With an At-Home Wearable.

Early development of gross motor skills is foundational for the upcoming neurocognitive performance. Here, we studied whether at-home wearable measurements performed by the parents could be used to quantify and track infants' developing motor abilities. Unsupervised at-home measurements of the infants' spontaneous activity were made repeatedly by the parents using a multisensor wearable suit (altogether 620 measurements from 134 infants at age 4-22months). Machine learning-based algorithms were developed to detect the reaching of gross motor milestones (GMM), to measure times spent in key postures, and to track the overall motor development longitudinally. Parental questionnaires regarding GMMs were used for developing the algorithms, and the results were benchmarked with the interrater agreement levels established by World Health Organization (WHO). A total of 97 infants were used for the algorithm development and cross-validation, whereas an external validation was done using 37 infants from an independent recruitment in the same hospital. The algorithms detected the reaching of GMMs very accurately (cross-validation: accuracy, 90.9%-95.5%; external validation, 92.4%-96.8%), which compares well with the human experts in the WHO reference study. The wearable-derived postural times showed strong correlation to parental assessments (ρ = .48-.81). Individual trajectories of motor maturation showed strong correlation to infants' age (ρ = .93). These findings suggest that infants' gross motor skills can be quantified reliably and automatically from unsupervised at-home wearable recordings. Such methodology could be used in health care practice and in all developmental studies for gaining real-world quantitation and tracking of infants' motor abilities.

Evaluation of Software-Optimized Protocols for Acoustic Noise Reduction During Brain MRI at 7 Tesla.

MR-generated acoustic noise may be particularly concerning at 7-Tesla (T) systems. Noise levels can be reduced by altering gradient output using software optimization. However, such alterations might influence image quality or prolong scan times, and these optimizations have not been well characterized. To evaluate image quality, sound pressure levels (SPLs), and perceived noise levels when using the acoustic noise reduction technique SofTone for T2-weighted fast spin echo (T2W FSE) and three-dimensional T1-weighted turbo field echo (3D T1W TFE), and to compare with conventional imaging during 7-T brain MRI. Prospective. Twenty-eight volunteers underwent brain MRI, with n = 26 for image quality evaluations. Conventional and SofTone versions of T2W FSE and 3D T1W TFE at 7 T. Peak SPLs (A-weighted decibels, dBA), participant-perceived noise levels (Borg CR10-scale), qualitative image assessments by three neuroradiologists (four-graded ordinal scales), interrater reliability, and percentage agreement. Paired t-test, Wilcoxon's Signed-Rank Test, and Krippendorff's alpha; p < 0.05 were considered statistically significant. SofTone significantly reduced peak SPLs: from 116.3 to 97.0 dBA on T2W FSE, and from 123.7 to 101.5 dBA on 3D T1W TFE. SofTone was perceived as significantly quieter than conventional scanning. T2W FSE showed no significant differences in image quality assessments (p = 0.21-1.00), except one radiologist noting significantly less artifact interference with SofTone. General image quality remained acceptable for 3D T1W TFE, though one radiologist scored it significantly lower with SofTone (mean scores: 3.08 vs. 3.65), and two radiologists observed significantly worse white and gray matter differentiation with SofTone (mean scores: 3.19 vs. 3.54; 2.27 vs. 2.81). SofTone can significantly reduce sound intensity and perceived noise levels while maintaining acceptable image quality with T2W FSE and 3D T1W TFE in brain MRI. It appears to be an effective tool for providing a safer, quieter 7-T scan environment. 4 Technical Efficacy: Stage 5.

Mapping immune checkpoint inhibitor side effects to item libraries for use in real-time side effect monitoring systems.

Monitoring for the side effects of novel therapies using patient-reported outcomes (PROs) is critical for ensuring patient safety. Existing static patient-reported outcome measures may not provide adequate coverage of novel side effects. Item libraries provide a flexible approach to monitoring for side effects using customized item lists, but the ideal process for matching side effects to items sourced from multiple item libraries is yet to be established. We sought to develop a pragmatic process for mapping side effects to items from three major item libraries using immune checkpoint inhibitor (ICI) side effects as an example. Using a consumer- and clinician-driven list of 36 ICI side effects, two authors independently mapped side effects to Common Terminology Criteria for Adverse Event (CTCAE) terms, and then to three item libraries: the Patient-Reported Outcome version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE), the European Organisation for Research and Treatment of Cancer (EORTC) Item Library, and the Functional Assessment of Chronic Illness Therapy (FACIT) searchable library. The rates of inter-rater agreement were recorded. Following item collation from the item libraries, we devised criteria for selecting the optimal item for each side effect for inclusion in a future electronic PRO system based on guidance from the above groups. All 36 side effects mapped to at least one CTCAE term, with eight mapping to more than one term. Twenty-three side effects mapped to at least one PRO-CTCAE term, 35 side effects mapped to at least one EORTC item, and 31 side effects mapped to at least one FACIT item. The inter-rater agreement rate was 100% (PRO-CTCAE), 83% (EORTC) and 75% (FACIT). Pre-determined criteria were applied to select the optimal item for each side effect from the three item libraries, producing a final 61-item list. Using ICI side effects as an example, we developed a pragmatic approach to creating customized item lists from three major item libraries to monitor for side effects of novel therapies in routine care. This process highlighted the challenges of using item libraries and priorities for future work to improve their usability.

How do emerging adults perceive negative role models in relation to the dimensional approach and personality traits? A qualitative study

Abstract This study examines the personality traits of negative role models of Turkish undergraduate psychology students. Semi-structured interviews were conducted with 50 participants regarding a specific negative role model they chose. Scripts were content analyzed and interpreted in terms of the Big Five Personality Model. Inter-rater reliability analysis indicated a high level of agreement between the two coders, showing an overall Cohen’s κ of .94. Results showed that negative role models were mainly described as having low levels of conscientiousness and agreeableness. Within the theme of conscientiousness negative role models' lack of motivation, desire for achievement or life purpose, indecisiveness, and inability to deal with life difficulties were prominent. Within the theme of agreeableness negative role models’ negative attitudes and behavior such as humiliating or dominating others were mentioned. Findings indicated that negative role models existed within the close social circle of the participant (i.e., being a family member), and they were described with some positive characteristics as well. Another finding showed that female participants’ negative role models were usually female, while male participants’ negative role models were male. Findings were discussed and recommendations were presented for future studies.

Reliability, validity, and diagnostic accuracy of the apathy evaluation scale in chronic stroke survivors

BackgroundWe aimed to determine the psychometric properties of the Apathy Evaluation Scale (AES) in chronic stroke survivors.MethodsIn this study, 112 non-cognitive impairment stroke survivors participated. Acceptability, inter-rater, and test-retest reliability of the three Persian versions of AES (clinician, informant, and self-rated) were evaluated. The correlation of three AES versions with the Hospital Anxiety and Depression Scale (HADS), Modified Rankin Scale (mRS), and Barthel Index (BI) was evaluated. To assess the diagnostic accuracy of the three AES versions, stroke survivors were classified as apathetic (n = 43) and non-apathetic (n = 69) groups using the ‘diagnostic criteria of apathy’.ResultsThe floor and ceiling effect, skewness, and kurtosis were within acceptable range for three AES versions. Internal consistency (α = 0.88–0.91) and test-retest and inter-rater reliability (ICC2,1 >0.90) were acceptable for all AES versions. Standard Error of Measurement and Minimal Detectable Change values for test-retest and/or inter-rater reliability ranged 1.6–2.5 and 4.42–6.93 for three versions of AES, respectively. Significant moderate to high correlation (r or ƿ = -0.34–0.69) was found between three AES versions and HADS-D, HADS-A, BI, and mRS. The cut-off point > 34 (sensitivity = 87.5%, specificity = 72.22%, and AUC = 0.80) was derived for discriminating apathetic from non-apathetic stroke survivors based on the AES- self-rated total score.ConclusionAll three AES versions are reliable and valid screening tools to evaluate and characterize apathy in stroke survivors. The AES-self-rated had good discriminative validity for discriminating apathetic from non-apathetic subjects in non-cognitive impairment stroke survivors.

Assessing ChatGPT’s Role in Sarcopenia and Nutrition: Insights from a Descriptive Study on AI-Driven Solutions

Background: Sarcopenia, an age-related decline in muscle mass and function, poses significant health risks. While AI tools like ChatGPT-4 (ChatGPT-4o) are increasingly used in healthcare, their accuracy in addressing sarcopenia remains unclear. Methods: ChatGPT-4’s responses to 20 frequently asked sarcopenia-related questions were evaluated by 34 experts using a four-criterion scale (relevance, accuracy, clarity, Ccmpleteness). Responses were rated from 1 (low) to 5 (high), and interrater reliability was assessed via intraclass correlation coefficient (ICC). Results: ChatGPT-4 received consistently high median scores (5.0), with ≥90% of evaluators rating responses ≥4. Relevance had the highest mean score (4.7 ± 0.5), followed by accuracy (4.6 ± 0.6), clarity (4.6 ± 0.6), and completeness (4.6 ± 0.7). ICC analysis showed poor agreement (0.416), with Completeness displaying moderate agreement (0.569). Conclusions: ChatGPT-4 provides highly relevant and structured responses but with variability in accuracy and clarity. While it shows potential for patient education, expert oversight remains essential to ensure clinical validity. Future studies should explore patient-specific data integration and AI comparisons to refine its role in sarcopenia management.

Development of an Intervention Population Ontology for specifying the characteristics of intervention participants

Background The uptake, effectiveness and generalisability of interventions are influenced by the features of the populations targeted. However, populations exposed to interventions are not consistently specified in published reports. Purpose To create an Intervention Population Ontology providing a clear, usable and reliable classification system to specify characteristics of populations exposed to interventions. Methods The Intervention Population Ontology was developed in seven main stages 1) Defining the ontology’s scope, (2) identifying key entities by reviewing existing classification systems (top-down) and 100 intervention reports (bottom-up), 3) Refining the preliminary ontology by annotating ~150 intervention reports, 4) Stakeholder review by 29 behavioural science and public health experts, 5) Assessing inter-rater reliability of using the ontology by two coders familiar with the ontology and two coders unfamiliar with it, 6) Specifying ontological relationships between entities in the ontology and 7) making the Intervention Population Ontology machine-readable using Web Ontology Language (OWL) and publishing online. Results The Intervention Population Ontology features 218 entities representing attributes of human individuals across 12 key groupings: personal attributes, geographic location, person, quality, mental capability, role, expertise, objects possessed, behaviour, personal vulnerability and personal history. It has a further 666 classes relating to how individual-level attributes are aggregated to describe groups of people. Inter-rater reliability was α=0.79 for coders familiar with the ontology and 0.85 for coders unfamiliar with the ontology. Conclusions The Intervention Population Ontology can be applied to specify precisely information from diverse sources, annotate population characteristics in existing intervention evaluation reports and guide future reporting.