Inter-rater Intraclass Correlation Research Articles

Reliability of the Italian Version of the Fugl-Meyer Upper Extremity Scale Administered Remotely.

Background/Objectives: Despite the increase in home-based rehabilitation, outcome measures for telerehabilitation are still underdeveloped. The Fugl-Meyer Assessment (FMA) is one of the most widely used tools for evaluating post-stroke motor deficits, with the upper extremity component (FMA-UE) recommended for assessing motor deficits of the arm. This study aims to examine the intrarater and interrater reliability of the Italian version of the FMA-UE, administered remotely via video conferencing during a robotic telerehabilitation program. Methods: Twenty stroke patients participated and underwent 20 sessions of remote upper limb rehabilitation with a robotic device. In-person evaluations were conducted before (T0) and after (T1) treatment, with additional remote assessments throughout. The study evaluated both intrarater and interrater reliability using Intraclass Correlation Coefficients (ICC) and Bland-Altman plots, classifying reliability as excellent for scores above 0.90. Results: Bland-Altman analysis showed no systematic variance for both intrarater and interrater reliability of the FMA-UE scale. Excellent reliability was found with intrarater ICC = 0.972 and interrater ICC = 0.981. Sections A and C of the FMA-UE showed excellent intrarater reliability, while sections B and D had satisfactory results. Both intrarater and interrater reliability analysis of the total score of the FMA-UE scale also showed strong agreement with Cohen's Kappa values above 0.70. Conclusions: The findings suggest that the remote administration of the FMA-UE scale is a reliable tool for assessing upper limb motor function in stroke patients, supporting its use in telerehabilitation settings.

Evaluating Quality and Readability of AI-generated Information on Living Kidney Donation.

The availability of high-quality and easy-to-read informative material is crucial to providing accurate information to prospective kidney donors. The quality of this information has been associated with the likelihood of proceeding with a living donation. Artificial intelligence-based large language models (LLMs) have recently become common instruments for acquiring information online, including medical information. The aim of this study was to assess the quality and readability of artificial intelligence-generated information on kidney donation. A set of 35 common donor questions was developed by the authors and used to interrogate 3 LLMs (ChatGPT, Google Gemini, and MedGPT). Answers were collected and independently evaluated using the CLEAR tool for (1) completeness, (2) lack of false information, (3) evidence-based information, (4) appropriateness, and (5) relevance. Readability was evaluated using the Flesch-Kincaid Reading Ease Score and the Flesch-Kincaid Grade Level. The interrater intraclass correlation was 0.784 (95% confidence interval, 0.716-0.814). Median CLEAR scores were ChatGPT 22 (interquartile range [IQR], 3.67), Google Gemini 24.33 (IQR, 2.33), and MedGPT 23.33 (IQR, 2.00). ChatGPT, Gemini, and MedGPT had mean Flesch-Kincaid Reading Ease Scores of 37.32 (SD = 10.00), 39.42 (SD = 13.49), and 29.66 (SD = 7.94), respectively. Using the Flesch-Kincaid Grade Level assessment, ChatGPT had an average score of 12.29, Gemini had 10.63, and MedGPT had 13.21 (P < 0.001), indicating that all LLMs had a readability at the college-level education. Current LLM provides fairly accurate responses to common prospective living kidney donor questions; however, the generated information is complex and requires an advanced level of education. As LLMs become more relevant in the field of medical information, transplant providers should familiarize themselves with the shortcomings of these technologies.

Reliability of ultrasound assessment of the rectus femoris muscle thickness: intra-rater, inter-rater, and inter-day analysis accounting for age and sex

BackgroundThe rectus femoris muscle plays an important role in maintaining lower limb stability and biomechanical control during basic physical activities. Both reduced quadriceps strength and decreased muscle thickness of the rectus femoris, as measured by ultrasound, are associated with an increased risk of falls in older adults. We estimated the relative and absolute intra-rater, inter-rater, and inter-day reliability of the procedure.MethodsA female biologist and a male physical educator, both holding a master’s degree in human movement and rehabilitation, captured ultrasound images of the right rectus femoris muscle in 106 healthy participants (58 females and 48 males), aged 18 to 73 years. Each rater captured three images per participant during each visit, with two visits 7 to 10 days apart. A third person subsequently measured the muscle thickness. We calculated the Intraclass Correlation Coefficients (ICC) using a two-way random model and determined the 95% minimal detectable difference (MDD).ResultsThe mean muscle thickness was 2.12 cm. The reliability based on single measurements was as follows: the intra-rater ICC for raters A and B was 0.998 at both visits (MDDs: 0.074–0.082 cm). The inter-rater ICC was 0.976 at visit 1 and 0.977 at visit 2 (MDDs: 0.269–0.270 cm). The inter-day ICC was 0.973 for rater A and 0.972 for rater B (MDDs: 0.286–0.291 cm). Sensitivity analyses accounting for age, sex, and the use of mean measurements produced similar results. The results were homoscedastic.ConclusionsThe rectus femoris muscle thickness was measured with good reliability using ultrasonography across all the settings.

Assessing the Elasticity of the Flexor Pronator Muscles After Throwing Using Ultrasound Shear Wave Elastography.

The medial ulnar collateral ligament (UCL) of the elbow joint is the primary restraint to valgus stress during the throwing motion. The flexor pronator muscles (FPMs) also stabilize the elbow joint against valgus forces; however, assessment of FPM stiffness in baseball players has been limited. To use ultrasound shear wave elastography (USWE) to evaluate the change in tissue elasticity of the FPMs due to pitching. Descriptive laboratory study. Included were 14 healthy male amateur baseball players (mean age, 28.0 ± 2.6 years) who had played intermediate-level high school or college baseball. The tissue elasticity of the flexor digitorum superficialis (FDS) in the superficial layer of the FPMs and the flexor digitorum profundus (FDP) in the deep layer of the FPMs were measured using USWE before, immediately after, and 24 hours after a throwing session of 100 pitches. The intra- and interrater reliability of the measurements were evaluated using the Pearson correlation coefficient and intraclass correlation coefficient (ICC). The elasticity of each muscle was analyzed using repeated-measures analysis of variance followed by post hoc analysis. Reliability of the USWE measurements was good to excellent (intrarater ICCs, 0.78-0.94; interrater ICCs, 0.85-0.96). The mean tissue elasticity values before, immediately after, and 24 hours after pitching were 22.3 ± 4.4, 41.0 ± 13.8, and 38.3 ± 11.2 kPa, respectively, for the FDS and 27.1 ± 5.8, 48.0 ± 22.3, and 29.6 ± 11.5 kPa, respectively, for the FDP. For both the FDS and FDP, elasticity was significantly higher immediately after pitching than before pitching (P < .001 and P = .0027, respectively). While the elasticity of the FDS at 24 hours after pitching remained significantly higher compared with before pitching (P = .0011), the elasticity of the FDP at 24 hours decreased to the same level as before pitching (P = .91). Results using USWE indicated that the elasticity of the FDS did not decrease 24 hours after pitching and remained significantly higher than before pitching, suggesting that it may not be fully functioning as a dynamic stabilizer during this period. Pitching in such a condition may result in greater stress on the UCL and increase the risk of UCL injury.

Test-Retest and Inter-Rater Reliability for Selected Outcomes from a Wearable 3D Inertial Sensor Over Different Stable and Unstable Postural Conditions: A Validation Study

ABSTRACT The agreement between a wearable inertial sensor (GYKO, G) and the force platform (P) was assessed by evaluating test-retest and inter-rater reliability. Thirty-eight subjects were enrolled; the selected indices of balance were investigated over foot positions and (un)stable conditions. Intraclass correlation coefficient (ICC), standard error of measurement (SEM%) and minimal detectable change (MDC95%) were computed. For G, ICC bounds range from poor to excellent (0.040 ÷ 0.921), mean of 0.687 (= moderate reliability). Regarding P, ICC ranges from poor to excellent (0.070 ÷ 0.920), mean of 0.683 (= moderate reliability). For G, SEM% ranges from 11% to 47%; MDC95% from 30% to 132%. Concerning P, SEM% ranges from 7% to 41%; MDC95% from 21% to 114%. Finally, the inter-rater reliability ICC by comparing devices ranges from poor to excellent (−0.162 ÷ 0.911), mean of 0.338 (= poor reliability). GYKO appears to be a convenient tool with high consistency among multiple measurements but for specific clinic/research purposes.

Reliability of Rehabilitative Musculoskeletal Sonography for Measuring the Visible Cross-Sectional Area of Suboccipital Muscles.

Aims The primary aim of this study is to assess the reliability of rehabilitative sonography as a non-invasive imaging technique for measuring the visible cross-sectional area (CSA) of the deep suboccipital muscles. Objectives Objectives involve determining both the intra-rater and inter-rater reliability of sonographic measurements to ensure consistent and reproducible results across different sessions and examiners. The ultimate goal is to validate rehabilitative sonography as a reliable tool for clinical and research applications in the assessment and management of musculoskeletal (MSK) conditions involving the deep suboccipital muscles. Design and setting Seventeen participants, including nine women and eight men without neck or vertebral pain, were evaluated using MSK sonography. The visible CSA of suboccipital muscles was assessed using real-time B-mode sonography in a sitting position with a linear probe (5 cm, 7.5 MHz) aligned perpendicularly to the muscle fibers of the rectus capitis posterior minor (RCPM) in one sonogram and the obliquus capitis inferior (OCI) and rectus capitis posterior major (RCPMJ) in another. Two experienced examiners performed sonography measurements, and the procedure was repeated one hour and again one day later. Results Intra-class correlation coefficient (ICC) values for Examiner 1 were 0.84 and 0.79 for RCPM, 0.84 and 0.71 for RCPMJ, and 0.86 and 0.77 for OCI at the one-hour and one-day repetitions, respectively. ICC values for Examiner 2 were 0.86 and 0.77 for RCPM, 0.77 and 0.86 for RCPMJ, and 0.61 and 0.64 for OCI at the one-hour and one-day repetitions, respectively. Inter-rater ICC values were 0.77 for RCPM, 0.82 for RCPMJ, and 0.68 for OCI. Standard error of measurement (SEM) values for Examiner 1 were 11.63 and 13.92 mm² for RCPM, 10.30 and 7.91 mm² for RCPMJ, and 22.84 and 34.61 mm² for OCI. SEM values for Examiner 2 were 11.82 and 13.40 mm² for RCPM, 11.91 and 7.04 mm² for RCPMJ, and 39.20 and 37.73 mm² for OCI. SEM values between examiners were 13.74 mm² for RCPM, 10.36 mm² for RCPMJ, and 36.03 mm² for OCI. Conclusions These findings suggest that sonography is a reliable method for measuring the CSA of the RCPM, RCPMJ, and OCI muscles, provided that the examiners are well-trained and consistent in their techniques and that the average of three measurements is used.

Development and validation of the Oral Lichen Planus-Disease Activity Scale: a novel tool for comprehensive clinical assessment.

To develop and validate the Oral Lichen Planus-Disease Activity Scale (OLP-DAS) for assessing overall disease activity of OLP. The OLP-DAS was created by refining the Thongprasom criteria, incorporating inputs from the literature and expert review, and integrating pain assessment. Content validity was evaluated in a virtual meeting with 8 Oral Medicine specialists. Reliability and validity of the final version were examined. Seventeen OLP subjects were assessed for disease activity by 10 investigators using the OLP-DAS, Oral Disease Severity Score (ODSS), OLP-Investigator Global Assessment (OLP-IGA), and Reticular-Erythema-Ulcerative (REU) scale. Convergent validity was assessed by rating 160 OLP subjects using the OLP-DAS, ODSS, and OLP-IGA. Inter-rater and intra-rater reliability, along with convergent validity, were analyzed using intraclass correlation coefficients (ICCs) and Spearman's rank correlation coefficients (rs). The final OLP-DAS achieved excellent content validity indices. Inter-rater and intra-rater ICCs for total OLP-DAS scores were 0.93 and 0.96, respectively. Total OLP-DAS scores exhibited strong positive correlations with the ODSS and OLP-IGA (rs = 0.94 and rs = 0.76; P < 0.001, respectively). The OLP Severity Index (OLP-SI), a component of the OLP-DAS, showed very strong positive correlations with OLP disease activity parameters of the ODSS (rs = 0.90; P < 0.001). The OLP-DAS is a valid and reliable clinician-reported outcome measure (CROM) for evaluating OLP disease activity. The OLP-DAS, as a standardized CROM for OLP, is valuable for both routine clinical assessments and research applications.

Performance of bowel preparation quality scales in patients with Crohn's disease.

The performance of bowel preparation (BP) in patients with Crohn's disease (CD) is unknown. To evaluate the operating properties of instruments used to assess BP quality in patients with CD. We used the Boston Bowel Preparation Scale, modified Boston Bowel Preparation Scale, Harefield Cleansing Scale, Food and Drug Administration Bowel Cleansing Assessment Scale (BCAS), and a 100-mm visual analogue scale of bowel cleanliness to assess BP quality in 50 videos from 40 patients with CD. We assessed endoscopic activity with the Simple Endoscopic Score for CD (SES-CD). Assessments were on endoscope insertion and withdrawal. Reliability was quantified using the intraclass correlation coefficient (ICC). We assessed validity by within-patient correlation between instruments and the visual analogue scale using mixed-effect models. The correlation between BP quality and SES-SD scores was assessed using Spearman's rho. Inter- and intra-rater reliability for all BP quality instruments was substantial (ICC ≥0.61) except for the Food and Drug Administration BCAS on insertion (inter-rater reliability ICC ≥0.41). The visual analogue scale had substantial inter- and almost perfect (ICC ≥0.81) intra-rater reliability. Correlation coefficients for the validity of the instruments exceeded 0.58. BP quality and endoscopic disease activity scores in the colon were negatively correlated. Most existing instruments reliably assess BP quality in patients with CD. These results support the use of these instruments in clinical practice, provide a framework for scoring BP quality in CD clinical trials, and support evaluation of novel BP agents in patients with CD.

Intra- and interrater measurement reliability of lateral ankle ligament attachment locations using three-dimensional magnetic resonance imaging

BackgroundWe aimed to evaluate the intra- and interrater measurement reliability of the lateral ankle ligament attachment locations using three-dimensional magnetic resonance imaging. MethodsWe analysed 54 participants with a mean age of 43 years who underwent three-dimensional ankle magnetic resonance imaging and had normal lateral ligaments. Bony landmarks of the distal fibula, talus, and calcaneus were identified in the reconstructed images. The centers of the anterior talofibular ligament and calcaneofibular ligament attachments were also identified. The distances between the landmarks and attachments were measured. Two raters performed the measurements twice, and intra- and interrater intraclass correlation coefficients were calculated. ResultsThe intrarater intraclass correlation coefficient values were between 0.71 and 0.96 for the anterior talofibular ligament attachment measurements and between 0.77 and 0.95 for the calcaneofibular ligament attachments. The interrater intraclass correlation coefficient was higher than 0.7, except for the distance between the anterior talofibular ligament superior bundle and fibular obscure tubercle. The fibular attachment of a single-bundle anterior talofibular ligament was located 13.3 mm from the inferior tip and 43% along the anterior edge of the distal fibula. The superior and inferior bundles of the double-bundle ligament were located at 43% and 23%, respectively. The calcaneofibular ligament fibular attachment was 5.5 mm from the inferior tip, at 16% along the anterior edge of the distal fibula. ConclusionThe measurements of anterior talofibular ligament and calcaneofibular ligament attachment locations identified on three-dimensional magnetic resonance imaging were sufficiently reliable. This measurement method provides in vivo anatomical data on the lateral ankle ligament anatomy.

Reliability of CT Enterography for Describing Fibrostenosing Crohn Disease.

Background Standardized methods to measure and describe Crohn disease strictures at CT enterography are needed to guide clinical decision making and for use in therapeutic studies. Purpose To assess the reliability of CT enterography features to describe Crohn disease strictures and their correlation with stricture severity. Materials and Methods A retrospective study was conducted in 43 adult patients with symptomatic terminal ileal Crohn disease strictures who underwent standard-of-care CT enterography at a tertiary care center at the Cleveland Clinic between January 2008 and August 2016. After training on standardized definitions, four abdominal radiologists blinded to all patient information assessed imaging features (seven continuous measurements and nine observations) of the most distal ileal stricture in two separate sessions (separated by ≥2 weeks) in random order. Features with an interrater intraclass correlation coefficient (ICC) of 0.41 or greater (ie, moderate reliability or better) were considered reliable. Univariable and multivariable linear regression analysis identified reliable features associated with a visual analog scale of overall stricture severity. Significant reliable features were assessed as components of a CT enterography-based model to quantitate stricture severity. Results Examinations in 43 patients (mean age, 52 years ± 16 [SD]; 23 female) were evaluated. Five continuous measurements and six observations demonstrated at least moderate interrater reliability (interrater ICC range, 0.42 [95% CI: 0.25, 0.57] to 0.80 [95% CI: 0.67, 0.88]). Of these, 10 were univariably associated with stricture severity, and three continuous measurements-stricture length (interrater ICC, 0.64 [95% CI: 0.42, 0.81]), maximal associated small bowel dilation (interrater ICC, 0.80 [95% CI: 0.67, 0.88]), and maximal stricture wall thickness (interrater ICC, 0.50 [95% CI: 0.34, 0.62])-were independently associated (P value range, <.001 to .003) with stricture severity in a multivariable model. These three measurements were used to derive a well-calibrated (optimism-adjusted calibration slope = 1.00) quantitative model of stricture severity. Conclusion Standardized CT enterography measurements and observations can reliably describe terminal ileal Crohn disease strictures. Stricture length, maximal associated small bowel dilation, and maximal stricture wall thickness are correlated with stricture severity. © RSNA, 2024 Supplemental material is available for this article. See also the article by Rieder et al in this issue. See also the editorial by Galgano and Summerlin in this issue.

Periocular Asymmetry Index in Caucasian Populations Using Three-dimensional Photogrammetry Assessment

ObjectiveTo quantitatively assess the periocular asymmetry and investigate its sex and age-related differences in a Caucasian population using three-dimensional (3D) stereophotogrammetry.MethodStandardized 3D photos of the periocular region of 301 Caucasians were taken using the VECTRA M3 3D Imaging System. Standardized landmarks were positioned, and data measurements in the periocular region were obtained from these images using VAM software and assessed using intraclass correlation coefficients (ICC) for reliability. Absolute differences and asymmetry index between the right and the left periocular region were calculated with a focus on gender and age-related differences.ResultFissure height asymmetry was positively correlated with age (p < 0.01). Furthermore, reflex distance asymmetry was positively correlated with age in the elderly group (p < 0.05). Absolute differences of upper eyelid crease were 0.65 ± 0.79 mm for females and 0.90 ± 0.94 mm for males (p < 0.05). In the elderly group, the absolute differences of reflex distance were 0.664 ± 0.471 mm for males and 0.470 ± 0.408 mm for females (p < 0.05), while the absolute differences of fissure height were 0.619 ± 0.469 mm and 0.436 ± 0.372 mm, respectively (p < 0.05). All measurements had an inter-rater ICC and intra-rater ICC between 0.761 and 0.957.ConclusionPeriocular asymmetry is more prominent in older and male people. 3D photogrammetry is a reliable tool to detect periocular asymmetry and might provide an accurate and effective reference for periocular cosmetic, plastic, and reconstructive surgery in the clinical routine.Level of Evidence I This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors. www.springer.com/00266.

Assessing the interrater and intrarater reliability of subglottic stenosis grading systems

BackgroundSubglottic stenosis (SGS) is a condition leading to narrowing of the upper airway which can lead to dyspnea and life-threatening airway obstruction. Although other proposed grading systems exist, the Cotton Myer (CM) and percent stenosis systems are the most widespread in clinical practice. Despite this, the CM system has not yet been validated for visual assessment of SGS. ObjectiveTo determine the interrater and intrarater reliability of the CM grading system among a cohort of physicians who manage patients with SGS. MethodsAn online survey created with videos of tracheoscopies from 20 adult patients with subglotticstenosis (SGS) was sent individually to 9 expert physicians from various medical specialties, all of whom managed patients with SGS. Physicians were asked to view the 20 tracheoscopy videos and assess both the percent stenosis and Cotton Myer (CM) grade of each patient. After a period of 4 weeks, the physicians were sent the same survey of the 20 tracheoscopy videos. The interrater and intrarater reliability was calculated using the intraclass correlation coefficient (ICC), a measurement used to evaluate the reliability (the extent to which a measurement can be replicated) of two or more raters measuring the same subject. ResultsOverall, CM and percent stenosis systems were found to have an ICC of 0.94 and 0.90 within the domain of interrater reliability, respectively, and ICC of 0.71 and 0.81 within the domain of intrarater reliability, respectively. ConclusionOur findings suggest that the CM and percent stenosis grading systems remain a valid clinical tool to measure and communicate the severity of airway obstruction in SGS.

Analyse systématique des évaluations de circuits multiples d’examen clinique objectif structuré (ECOS) : variables explicatives et corrélations inter-évaluateurs

IntroductionObjective Structured Clinical Examinations (OSCEs) assess professional performance in a simulated environment. Following their integration into the reform of the 2nd cycle of medical studies (R2C), this pedagogical modality was implemented in France. This study investigates the variability of students’ OSCE scores, as well as their inter-rater reproducibility. MethodsThis single-center retrospective study covered several sessions of evaluative OSCE circuits conducted between January 2022 and June 2023. Variables collected were: baseline situation family, competency domain, presence of a standardized participant for stations; gender and professional status for evaluators; scores (global, clinical and communication skills), number of previously completed OSCE circuits and faculty scores for students. ResultsThe variability of the overall score was explained mainly (79.7%, CI95% [77.4; 82.0]) by the station factor. The student factor and the circuit factor explained 7.5% [12.9; 20.2] and<0.01% [2.10-13; 2.10-9] respectively. The inter-rater intra-class correlation coefficient was 87.2% [86.4; 87.9] for the global score. Station characteristics (starting situation, domain) and evaluator characteristics (gender, status) were significantly associated with score variations. ConclusionThis first study on the variability of OSCE circuit scores in France shows good reproducibility with influence of station characteristics. In order to standardize circuits, variability linked to the domain competency should be considered as well.

Testing the Reliability of Optical Coherence Tomography to Measure Epidermal Thickness and Distinguish Volar and Nonvolar Skin

In persons with limb loss, prosthetic devices cause skin breakdown, largely because residual limb skin (nonvolar) is not intended to bear weight such as palmoplantar (volar) skin. Before evaluation of treatment efficacy to improve skin resiliency, efforts are needed to establish normative data and assess outcome metric reliability. The purpose of this study was to use optical coherence tomography to (i) characterize volar and nonvolar skin epidermal thickness and (ii) examine the reliability of optical coherence tomography. Four orientations of optical coherence tomography images were collected on 33 volunteers (6 with limb loss) at 2 time points, and the epidermis was traced to quantify thickness by 3 evaluators. Epidermal thickness was greater (P < .01) for volar skin (palm) (265.1 ± 50.9 μm, n= 33) than for both nonvolar locations: posterior thigh (89.8 ± 18.1 μm, n= 27) or residual limb (93.4 ± 27.4 μm, n= 6). The inter-rater intraclass correlation coefficient was high for volar skin (0.887-0.956) but low for nonvolar skin (thigh: 0.292-0.391, residual limb: 0.211-0.580). Correlation improved when comparing only 2 evaluators who used the same display technique (palm: 0.827-0.940, thigh: 0.633-0.877, residual limb: 0.213-0.952). Despite poor inter-rater agreement for nonvolar skin, perhaps due to challenges in identifying the dermal-epidermal junction, this study helps to support the utility of optical coherence tomography to distinguish volar from nonvolar skin.

Measurement properties of the falls efficacy scale in patients on hemodialysis

BackgroundThe measurement properties of the falls efficacy scale have not been evaluated in patients on hemodialysis. This study determined the inter- and intrarater reliability, standard error of measurement, minimal detectable change, and limits of agreements of the falls efficacy scale in patients on hemodialysis.MethodsA cross-sectional study was conducted with 65 patients (57.5 ± 13.9 years, 63.1% male) on hemodialysis. The fear of falling was assessed by two previously trained raters using the falls efficacy scale. The intraclass correlation coefficient, standard error of measurement, minimal detectable change, and Bland–Altman plot were calculated to assess the inter- and intrarater reliability of the falls efficacy scale.ResultsThe interrater intraclass correlation coefficient was 0.91, and the intrarater intraclass correlation coefficient was 0.78, representing excellent interrater and good intrarater reliability. The standard error of measurement for inter- and intrarater assessments were 2.99 and 4.46, and the minimal detectable change for inter- and intrarater assessments were 9.26 and 12.33, respectively. The interrater mean difference score was 0.26 (95% limits of agreement: − 8.01 to 8.53), and the intrarater mean difference score was − 1.06 (95% limits of agreement: − 13.39 to 11.27).ConclusionIn patients on hemodialysis, the falls efficacy scale showed excellent and good inter- and intrarater reliability, respectively. Additionally, standard error of measurement, minimal detectable change, and limits of agreements of the falls efficacy scale score were satisfactory.

Spine surgeon versus AI algorithm full-length radiographic measurements: a validation study of complex adult spinal deformity patients.

Spinal measurements play an integral role in surgical planning for a variety of spine procedures. Full-length imaging eliminates distortions that can occur with stitched images. However, these images take radiologists significantly longer to read than conventional radiographs. Artificial intelligence (AI) image analysis software that can make such measurements quickly and reliably would be advantageous to surgeons, radiologists, and the entire health system. Institutional Review Board approval was obtained for this study. Preoperative full-length standing anterior-posterior and lateral radiographs of patients that were previously measured by fellowship-trained spine surgeons at our institution were obtained. The measurements included lumbar lordosis (LL), greatest coronal Cobb angle (GCC), pelvic incidence (PI), coronal balance (CB), and T1-pelvic angle (T1PA). Inter-rater intra-class correlation (ICC) values were calculated based on an overlapping sample of 10 patients measured by surgeons. Full-length standing radiographs of an additional 100 patients were provided for AI software training. The AI algorithm then measured the radiographs and ICC values were calculated. ICC values for inter-rater reliability between surgeons were excellent and calculated to 0.97 for LL (95% CI 0.88-0.99), 0.78 (0.33-0.94) for GCC, 0.86 (0.55-0.96) for PI, 0.99 for CB (0.93-0.99), and 0.95 for T1PA (0.82-0.99). The algorithm computed the five selected parameters with ICC values between 0.70 and 0.94, indicating excellent reliability. Exemplary for the comparison of AI and surgeons, the ICC for LL was 0.88 (95% CI 0.83-0.92) and 0.93 for CB (0.90-0.95). GCC, PI, and T1PA could be determined with ICC values of 0.81 (0.69-0.87), 0.70 (0.60-0.78), and 0.94 (0.91-0.96) respectively. The AI algorithm presented here demonstrates excellent reliability for most of the parameters and good reliability for PI, with ICC values corresponding to measurements conducted by experienced surgeons. In future, it may facilitate the analysis of large data sets and aid physicians in diagnostics, pre-operative planning, and post-operative quality control.

Reliability of the unified multiple system atrophy rating scale using the telephone

ObjectiveThe unified multiple system atrophy rating scale (UMSARS) was used to evaluate various symptoms of multiple system atrophy (MSA). And UMSARS part 1 was originally developed for use in interviews, but the need for telemedicine is increasing in COVID-19 pandemic. The purpose of this study is to evaluate the reliability of the UMSARS part 1 telephone survey. MethodsThirty-two MSA patients took the UMSARS part 1 face-to-face, followed by two more telephone evaluations. Intraclass correlation coefficients (ICC) and Cronbach’s alpha (α) coefficients were calculated, and the inter-rater reliability was determined. At the same time, we asked about the problems in COVID-19 pandemic. ResultsThe study participants included 15 men and 17 women with mean age of 67.1 years (SD, 8.3). For the total UMSARS part 1 score, the inter-rater ICC and Cronbach’s α coefficient were 0.89 to 0.92, and 0.84 to 0.87, respectively. More than half of the items had a relatively high ICC. Cronbach’s α coefficients were more than 0.7 for all items. Changes that occurred in COVID-19 pandemic included reduced outings and lack of rehabilitation in about half of the cases. ConclusionThe UMSARS part 1 has high inter-rater reliability and internal consistency. Evaluation of subjective symptoms showed that some variability could occur. In addition, there was concern about the influence of lack of rehabilitation due to COVID-19 pandemic.

In-person validation of the Ichthyosis Scoring System.

Ichthyoses are a heterogeneous group of skin disorders characterized by scaling and erythema. Recognizing the variability of scale and erythema by region and ichthyosis subtype, we developed the Ichthyosis Scoring System (ISS) to quantify severity. We previously found ISS to have high inter- and intrarater reliability in evaluating photographic images. To confirm ISS clinical utility, we examined its performance at the 2022 Foundation for Ichthyosis and Related Skin Types conference. Sixty-five participants were evaluated by 3 of 9 medical professionals trained to score ichthyosis scale and erythema using ISS. Intrarater and interrater intraclass correlation coefficients (ICC) were analyzed using one-way and two-way random effects models, respectively. Intrarater reliability was excellent (ICC = 0.931, 95% CI, 0.921-0.940) for scale and good (ICC = 0.876, 95% CI, 0.853-0.899) for erythema scoring. Compared to photo validation with excellent intrarater reliability ratings for both scale (ICC = 0.956, 95% CI, 0.925-0.974) and erythema (ICC = 0.913, 95% CI, 0.855-0.949), ISS demonstrated equivalent reliability for live use. Overall interrater reliability for 10 body sites showed excellent (ICC >0.9) and good (ICC >0.75) agreement and consistency for both scale and erythema. Palms were an exception, demonstrating moderate (ICC >0.5) interrater agreement and consistency for erythema evaluation. ISS is a reliable measure of global and regional ichthyosis severity during in-person evaluations. Ease-of-use, accessibility, and content validity in both live and photographic evaluation endorse ISS as a standard for ichthyosis severity analysis.

Ultrasound assessment of malnutrition in infancy: a pilot case-control study

This study aimed at determining the intra- and inter-rater reliability in ultrasound body composition measurements and investigating the differences between malnourished and non-malnourished infants. Sonographic images for measurements of fat and muscle thickness were compared between 9 malnourished and 9 non-malnourished hospitalized infants. The mean of fat and muscle thickness sums were 12.44 ± 7.58 mm and 28.98 ± 7.18 mm, respectively. The intra- and inter-rater intraclass correlation coefficient were above 0.9 for both measurements, indicating high intra- and inter-rater reliability. Compared to non-malnourished infants, malnourished infants have 45% of fat thickness sum and 71% of muscle thickness sum. Ultrasound measurements of body composition in infants were different between hospitalized malnourished and non-malnourished infants. This approach has the potential to be utilized more broadly, from assessing the nutritional status of critically ill infants in intensive care units to screening for malnutrition in high-risk infant populations.

Influence of chest wall conformation on reproducibility of main echocardiographic indices of left ventricular systolic function.

The possible influence of chest wall conformation, as noninvasively assessed by Modified Haller Index (MHI, the ratio of chest transverse diameter over the distance between sternum and spine), on reproducibility of both left ventricular ejection fraction (LVEF) and global longitudinal strain (GLS) has never been previously investigated. Two equal groups of healthy individuals, matched by age, sex, and cardiovascular risk factors and categorized according to MHI in those with concave-shaped chest wall (MHI>2.5) and those with normal chest shape (MHI≤2.5), who underwent transthoracic echocardiography implemented with echocardiographic deformation imaging between June 2018 and May 2019, were retrospectively analyzed. LVEF and GLS were measured twice by the two echocardiographers in a double blinded manner. Intra-class correlation coefficients (ICCs), bias and limits of agreement determined with Bland-Altman analysis were calculated for repeated measurements of both LVEF and GLS. Thirty-four healthy individuals with MHI>2.5 (54.9±6.4 years, 58.8% females) and 34 matched controls with MHI≤2.5 (52.5±8.1 years, 50% females) were separately analyzed. In comparison to MHI≤2.5 group, the MHI>2.5 group was found with significantly smaller cardiac chambers and significantly lower GLS magnitude (-15.8±2.5 vs. -22.2±1.3%, P<0.001), despite similar LVEF (61.3±6.4 vs. 61.1±3.6%, P=0.87). In the MHI>2.5 group, intra-rater and inter-rater ICCs were ≤0.5 for both LVEF and LV-GLS, whereas in the MHI≤2.5 group intra-rater and inter-rater ICCs values indicated good reliability for LVEF and excellent reliability for GLS. The greatest bias and largest limits of agreement were detected for LVEF assessment (bias ranging from -1.09 to 2.94%, with the 95% limits of agreement ranging from -13.9 to 21.3%) in individuals with MHI>2.5. On the other hand, the smallest bias and narrowest limits of agreement were obtained for GLS measurement (bias ranging from -0.26 to 0.09%, with the 95% limits of agreement ranging from -1.4 to 1.4%) in participants with normal chest wall conformation (MHI≤2.5). The test reliability of LVEF and GLS is strongly influenced by the chest wall conformation. MHI might represent an innovative approach for selecting the best echocardiographic method for LV systolic function estimation in the individual case.