Agreement Intraclass Correlation Research Articles

Comparison of semi-automatic and manual segmentation methods for tumor delineation on head and neck squamous cell carcinoma (HNSCC) positron emission tomography (PET) images

Objective. Accurate and reproducible tumor delineation on positron emission tomography (PET) images is required to validate predictive and prognostic models based on PET radiomic features. Manual segmentation of tumors is time-consuming whereas semi-automatic methods are easily implementable and inexpensive. This study assessed the reliability of semi-automatic segmentation methods over manual segmentation for tumor delineation in head and neck squamous cell carcinoma (HNSCC) PET images. Approach. We employed manual and six semi-automatic segmentation methods (just enough interaction (JEI), watershed, grow from seeds (GfS), flood filling (FF), 30% SUVmax and 40%SUVmax threshold) using 3D slicer software to extract 128 radiomic features from FDG-PET images of 100 HNSCC patients independently by three operators. We assessed the distributional properties of all features and considered 92 log-transformed features for subsequent analysis. For each paired comparison of a feature, we fitted a separate linear mixed effect model using the method (two levels; manual versus one semi-automatic method) as a fixed effect and the subject and the operator as the random effects. We estimated different statistics—the intraclass correlation coefficient agreement (aICC), limits of agreement (LoA), total deviation index (TDI), coverage probability (CP) and coefficient of individual agreement (CIA)—to evaluate the agreement between the manual and semi-automatic methods. Main results. Accounting for all statistics across 92 features, the JEI method consistently demonstrated acceptable agreement with the manual method, with median values of aICC = 0.86, TDI = 0.94, CP = 0.66, and CIA = 0.91. Significance. This study demonstrated that JEI method is a reliable semi-automatic method for tumor delineation on HNSCC PET images.

Reliability and measurement error of a maximal voluntary toe plantarflexion measurement process

Despite the growing interest, information regarding the psychometric properties of maximal voluntary isometric toe plantarflexion force and rate of force development (RFD) is lacking. Hence, we investigate the test-retest reliability and measurement error of these outcome measurement instruments measured with a custom-built dynamometer. Twenty-six healthy adults participated in a crossed design with four sessions separated by 5–7 days. RFD was quantified using manual onset and calculating the impulse and the slope in the following time windows: 0–50 ms, 0–100 ms, 0–150 ms, 0–200 ms, 0–250 ms. We estimated the systematic bias of the mean, the intraclass correlation coefficient (ICC) and standard error of measurement (SEM) from the agreement and consistency models. The ICC and the SEM agreement for maximal voluntary isometric toe plantarflexion force along the perpendicular axis were respectively 0.87 (95%CI: 0.76, 0.93) and 27 N (22, 32), while along the resultant of the perpendicular and anterior posterior axis they were 0.85 (0.73, 0.92) and 29 N (23, 35). The results of the consistency model were similar as the estimated variance for session was closer to zero. A systematic bias of the mean between session 1 and 3 was found. For the RFD variables, the ICC agreement ranged from 0.35 to 0.65. The measurement process was found to be reliable to assess maximal voluntary isometric toe plantarflexion force but not RFD. However, a familiarization session is mandatory and these results need to be confirmed in less coordinated (e.g. aging population) individuals.

Further development and validation of the Multimorbidity Treatment Burden Questionnaire (MTBQ)

ObjectivesTo undertake further psychometric testing of the Multimorbidity Treatment Burden Questionnaire (MTBQ) and examine whether reversing the scale reduced floor effects.DesignSurvey.SettingUK primary care.ParticipantsAdults (≥18 years) with three or more long-term...

Quantitative MRI assessment of joint effusion using T2-relaxometry at 3 Tesla: a feasibility and reproducibility study.

T2-relaxometry could differentiate between physiological and haemorrhagic joint effusion (≥ 5% blood) in vitro. Are quantitative T2-relaxation time measurements of synovial fluid feasible and reproducible in vivo in clinically bleed-free joints of men with haemophilia? In this cross-sectional study, we measured T2-relaxation times of synovial fluid in clinically bleed-free ankles, knees or elbows of men with severe haemophilia A using a T2-mapping sequence (duration ≤ 7min) at 3 Tesla MRI. Manual and circular regions of interest (ROI) were drawn in the synovial fluid of each joint by two independent observers to measure T2-relaxation times. Measurement feasibility was expressed as the success rate of the measurements by both observers. The interobserver and intraobserver reproducibility of the measurements were evaluated by the intraclass correlation coefficient of absolute agreement (ICC) and the limits of agreement (LoA) from Bland Altman analysis. We evaluated 39 clinically bleed-free joints (11 ankles, 12 knees, 16 elbows) of 39 men (median age, 24years; range 17-33) with severe haemophilia A. The success rate of the T2-measurements was ≥ 90%. Interobserver reliability was good to excellent (manual ROI: ICC = 0.92, 95% CI 0.76-0.97; circular ROI: ICC = 0.82, 95% CI 0.66-0.91) and interobserver agreement was adequate (manual ROI: LoA = 71ms; circular ROI: LoA = 146ms). Intraobserver reliability was good to excellent (manual ROI: ICC = 0.78, 95% CI - 0.06-0.94; circular RO:ICC = 0.99, 95% CI 0.98-0.99) and intraobserver agreement was good (manual ROI: LoA = 63ms; circular ROI: LoA = 41ms). T2-relaxometry of synovial fluid in haemophilia patients is feasible with good interobserver and intraobserver reproducibility.

Comparison of injury severity scores derived from ICD-10-AM codes with trauma registry derived scores: A study from New Zealand

IntroductionVarious attempts at automation have been made to reduce the administrative burden of manually assigning Abbreviated Injury Severity (AIS) codes to derive Injury Severity Scores (ISS) in trauma registry data. The accuracy of the resulting measures remains unclear, especially in the New Zealand (NZ) context. The aim of this study was to compare ISS derived from hospital discharge International Classification of Diseases Australian Modification (ICD-10-AM) codes with ISS recorded in the NZ Trauma Registry (NZTR). MethodsIndividuals admitted to hospital and enrolled in the NZTR between 1 December 2016 and 30 November 2018 were included. ISS were calculated using a modified ICD to AIS mapping tool. The agreement between both methods for raw scores was assessed by the Intraclass Correlation Coefficient (ICC), and for categorical scores the Kappa and weighted Kappa index were used. Analysis was conducted by gender, age, ethnicity, and mechanism of injury. Results3,156 patients fulfilled the inclusion criteria. The ICC for agreement between the methods was poor (0.40, 95 % CI: 0.37–0.43). The Kappa index indicated slight agreement between both methods when using a cut-off value of 12 (0.06; 95 % CI: 0.01–0.12) and 15 (0.13 6; 95 % CI: 0.09–0.17). ConclusionAlthough the overall agreement between NZTR-ISS and ICD-ISS was slight, ICD-derived scores may be useful to describe injury patterns and for body region-specific estimations when manually coded ISS are not available.

Novel AI-Based Algorithm for the Automated Measurement of Cervical Sagittal Balance Parameters. A Validation Study on Pre- and Postoperative Radiographs of 129 Patients.

Retrospective, mono-centric cohort research study. The analysis of cervical sagittal balance parameters is essential for preoperative planning and dependent on the physician's experience. A fully automated artificial intelligence-based algorithm could contribute to an objective analysis and save time. Therefore, this algorithm should be validated in this study. Two surgeons measured C2-C7 lordosis, C1-C7 Sagittal Vertical Axis (SVA), C2-C7-SVA, C7-slope and T1-slope in pre- and postoperative lateral cervical X-rays of 129 patients undergoing anterior cervical surgery. All parameters were measured twice by surgeons and compared to the measurements by the AI algorithm consisting of 4 deep convolutional neural networks. Agreement between raters was quantified, among other metrics, by mean errors and single measure intraclass correlation coefficients for absolute agreement. ICC-values for intra- (range: .92-1.0) and inter-rater (.91-1.0) reliability reflect excellent agreement between human raters. The AI-algorithm could determine all parameters with excellent ICC-values (preop:0.80-1.0; postop:0.86-.99). For a comparison between the AI algorithm and 1 surgeon, mean errors were smallest for C1-C7 SVA (preop: -.3mm (95% CI:-.6 to -.1mm), post: .3mm (.0-.7mm)) and largest for C2-C7 lordosis (preop:-2.2° (-2.9 to -1.6°), postop: 2.3°(-3.0 to -1.7°)). The automatic measurement was possible in 99% and 98% of pre- and postoperative images for all parameters except T1 slope, which had a detection rate of 48% and 51% in pre- and postoperative images. This study validates that an AI-algorithm can reliably measure cervical sagittal balance parameters automatically in patients suffering from degenerative spinal diseases. It may simplify manual measurements and autonomously analyze large-scale datasets. Further studies are required to validate the algorithm on a larger and more diverse patient cohort.

Validation of algorithms for calculating spatiotemporal gait parameters during continuous turning using lumbar and foot mounted inertial measurement units

Spatiotemporal gait parameters such as step time and walking speed can be used to quantify gait performance and determine physical function. Inertial measurement units (IMUs) allow for the measurement of spatiotemporal gait parameters in unconstrained environments but must be validated against a gold standard.While many IMU systems and algorithms have been validated during treadmill walking and overground walking in a straight line, fewer studies have validated algorithms during more complex walking conditions such as continuous turning in different directions.This study explored the concurrent validity in a population of healthy adults (range 26–52 years) of three different algorithms using lumbar and foot mounted IMUs to calculate spatiotemporal gait parameters: two methods utilizing an inverted pendulum model, and one method based on strapdown integration. IMU data was compared to a Vicon twelve-camera optoelectronic system, using data collected from 9 participants performing straight walking and continuous walking trials at different speeds, resulting in 162 walking trials in total. Intraclass correlation coefficients (ICCA,1) for absolute agreement were calculated between the algorithm outputs and Vicon output.Temporal parameters were comparable in all methods and ranged from moderate to excellent, except double support time which was poor. Strapdown integration performed better for estimating spatial parameters than pendulum models during straight walking, but worse during turning. Selecting the most appropriate model should take into consideration both speed and walking condition.

Concurrent Validity and Reliability of In-Field Vertical Jump Performance Measures on Sand Surfaces.

Wirtz, S, Julian, R, Schmale, R, and Eils, E. Concurrent validity and reliability of in-field vertical jump performance measures on sand surfaces. J Strength Cond Res 38(4): 687-693, 2024-Beach sports, such as handball, volleyball, and soccer, involve dynamic actions, primarily jumps. Nonetheless, there is a need for more established measurement devices that are both valid and feasible in accurately evaluating jumping performance on sand surfaces. This study aims to assess the reliability and concurrent validity of MyJump2 and an inertial measurement unit (VERT) for quantifying countermovement jump performance on the sand. Twenty-one subjects performed 5 countermovement jumps on the sand. Jumping height was measured simultaneously by a force plate, VERT, and MyJump2. The VERT and MyJump2 reliability measures were evaluated using the intraclass correlation coefficients (ICC) for both absolute agreement (ICC 2,1 ) and consistency (ICC 3.1 ). The day-to-day reliability of both devices and the interrater and intrarater reliability of the MyJump2 were assessed. The concurrent validity of the VERT and MyJump2 was assessed using ICCs 2.1, 3.1 and Bland-Altman plots. A near-perfect agreement was seen for intrarater (ICC = 0.98) and interrater (ICC = 0.98) reliability of the MyJump2. Moreover, a near-perfect agreement was also observed when comparing the MyJump2 with the force plate (ICC = 0.92), with a mean bias of -2.31 cm. The VERT device only demonstrated a moderate agreement (ICC = 0.55) with a mean bias of 6.31 cm compared with the force plate, despite showing good day-to-day reliability (ICC = 0.79). The study's results indicate that MyJump2 is valid and reliable for assessing jump heights on sand surfaces, whereas VERT cannot be recommended.

Assessing the robustness of muscle strength and physical performance measures in women older than 40 years: a test-retest reliability study.

This study investigated the test-retest reliability and minimal detectable change (MDC) of muscle strength and physical performance tests in women older than 40 years. A total of 113 women, aged 58 ± 11 years (82% postmenopausal women), with a body mass index of 28.5 ± 5.8 kg/m 2 , participated in this study. One-repetition maximum (1RM) tests were conducted for seven strength exercises: 45° leg press, bench press, leg extension, pec deck, wide-grip lateral pull-down, leg curl, and seated cable row, with a retest after 48 hours. Timed Up and Go (TUG), 6-minute walk, and 30-second sit-to-stand tests were performed on the same day and retested after 72 hours. The tests and retests were administered by the same evaluators. Relative reliability (consistency of participant rank between test-retest) was assessed using the intraclass correlation coefficient for consistency and agreement, and absolute reliability (precision of score) was assessed using the MDC based on the standard error of prediction. The 1RM and performance tests exhibited excellent reliability: 45° leg press (consistency, 0.99; agreement, 0.98), bench press (consistency, 0.96; agreement, 0.96), leg extension (consistency, 0.93; agreement, 0.91), pec deck (consistency, 0.90; agreement, 0.88), wide-grip lateral pull-down (consistency, 0.91; agreement, 0.89), leg curl (consistency, 0.84; agreement, 0.83), seated cable row (consistency, 0.92; agreement, 0.94), TUG (consistency, 0.87; agreement, 0.87), 6-minute walk (consistency, 0.96; agreement, 0.95), and 30-second sit to stand (consistency, 0.85; agreement, 0.80). These tests showed meaningful MDC values, particularly for the 1RM test performed on the machine and the 30-second sit-to-stand test: 45° leg press, 22.2 kg (15%); bench press, 4.9 kg (19%); leg extension, 9.2 kg (35%); pec deck, 8.8 kg (44%); wide-grip lateral pull-down, 9.4 kg (31%); leg curl, 9.2 kg (51%); seated cable row, 11.4 kg (29%); TUG, 1.4 seconds (20%); 6-minute walk, 50.9 m (10%); and 30-second sit to stand, 4.6 reps (30%). Although the muscle strength and physical performance tests demonstrate excellent relative reliability in women older than 40 years, they exhibit low absolute reliability, particularly the 1RM test performed on the machine and the 30-second sit-to-stand test. Therefore, although these tests show good consistency in the positioning of women within the group after repeated measures, their precision measure is relatively low (high fluctuation). Changes in these tests that are smaller than the MDC may not indicate real changes in women at middle age and older.

Episiotomy: Are Indian Obstetricians Getting the Angle Right?

Abstract Background A mediolateral episiotomy is recommended when indicated at a 60° angle at crowning, to avoid obstetric anal sphincter injuries (OASIs) by episiotomies angled too close or distant to the anus. This study surveyed obstetricians in India regarding the recommended episiotomy angle and their ability to correctly draw the angle. Methods Workshops were conducted in India to share knowledge in the prevention and repair of OASIs. A questionnaire was distributed prior to the workshop. Participants were asked to describe the recommended episiotomy angle and to draw this on a paper replica of the perineum. The intra-class correlation coefficient (ICC) was calculated to assess the inter-rater reliability between the angle stated and drawn. A 2° difference was deemed acceptable. Standard errors of measurement (SEM) were calculated to measure the range of error of each measurement. Results One hundred and forty doctors participated. 47.9% described the angle of an episiotomy to be 60°. Only 2.2% drew an angle of 60°, but 8.7% (n = 12) drew between 58 and 62°. Only 5.8% (n = 6) of doctors correctly drew the episiotomy angle they described. There was poor agreement ICC = 0.18 (− 0.01 to 0.36) with a SEM of ± 12.2° Conclusions Knowledge surrounding the recommended episiotomy angle is lacking. Doctors are failing to estimate their desired episiotomy angle. This highlights the need for national guidelines, the creation and validation of structured training programmes to improve accuracy, or using fixed-angle devices such as the EPISCISSORS-60 or other proven measurement aids to minimise preventable harm due to human error.

Collecting evidence of validity for an assessment tool for Norwegian medical students’ non-technical skills (NorMS-NTS): usability and reliability when used by novice raters

BackgroundThe NorMS-NTS tool is an assessment tool for assessing Norwegian medical students’ non-technical skills (NTS). The NorMS-NTS was designed to provide student feedback, training evaluations, and skill-level comparisons among students at different study sites. Rather than requiring extensive rater training, the tool should capably suit the needs of busy doctors as near-peer educators. The aim of this study was to examine the usability and preliminary assess validity of the NorMS-NTS tool when used by novice raters.MethodsThis study focused on the usability of the assessment tool and its internal structure. Three raters used the NorMS-NTS tool to individually rate the team leader, a medical student, in 20 video-recorded multi-professional simulation-based team trainings. Based on these ratings, we examined the tools’ internal structure by calculating the intraclass correlation coefficient (ICC) (version 3.1) interrater reliability, internal consistency, and observability. After the rating process was completed, the raters answered a questionnaire about the tool’s usability.ResultsThe ICC agreement and the sum of the overall global scores for all raters were fair: ICC (3,1) = 0.53. The correlation coefficients for the pooled raters were in the range of 0.77–0.91. Cronbach’s alpha for elements, categories and global score were mostly above 0.90. The observability was high (95%-100%). All the raters found the tool easy to use, none of the elements were redundant, and the written instructions were helpful. The raters also found the tool easier to use once they had acclimated to it. All the raters stated that they could use the tool for both training and teaching.ConclusionsThe observed ICC agreement was 0.08 below the suggested ICC level for formative assessment (above 0.60). However, we know that the suggestion is based on the average ICC, which is always higher than a single-measure ICC. There are currently no suggested levels for single-measure ICC, but other validated NTS tools have single-measure ICC in the same range. We consider NorMS-NTS as a usable tool for formative assessment of Norwegian medical students’ non-technical skills during multi-professional team training by raters who are new to the tool. It is necessary to further examine validity and the consequences of the tool to fully validate it for formative assessments.

Deep-learning based analysis of intracoronary optical coherence tomography images: detection and characterization of lipid plaques

Abstract Background Intracoronary optical coherence tomography (OCT) enables in vivo detection and characterization of atherosclerotic disease. However, manual assessment of OCT images is time-consuming and subject to intra- and interobserver variability. Therefore, automated and trustworthy methods for plaque assessment are warranted. Purpose To develop and validate an artificial intelligence based algorithm for detection and characterization of lipid plaques on OCT images. Methods From the prospective, observational PECTUS-obs study, we studied a representative subset of OCT pullbacks performed in fractional flow reserve-negative non-culprit lesions in patients presenting with myocardial infarction. Manual segmentation of cross-sectional frames (e.g. lumen, tunica intima including the fibrous cap, tunica media and lipid pools) was performed by trained experts. Pullbacks were randomly divided into a training and test set. For automated segmentation and analysis, a two-dimensional no-new U-shaped neural network (nnUNet) was constructed with 5-fold cross validation. To test the diagnostic performance of the nnUnet for detection of lipid plaques, sensitivity, specificity and Cohen’s kappa were calculated. As for plaque characterization, the lipid arc and minimal fibrous cap thickness were measured manually in the test set on each frame containing a lipid plaque. Values of lipid arc and minimal fibrous cap thickness obtained following automated assessment were compared to manual analysis using the intraclass correlation coefficient (ICC) for absolute agreement. Results In total, 1215 frames were used for training and 162 frames for testing of the nnUNet. Lipid plaques were present in 77 frames (47.5%) in the test set. Substantial agreement (κ=0.79) for detection of a lipid plaque was achieved with a sensitivity and specificity of 96.1% and 83.5%, respectively. As for plaque characterization, we found good reliability for lipid arc assessment (ICC 0.88, mean difference 6±32º) and moderate reliability for assessment of the minimal fibrous cap thickness (ICC 0.54, mean difference 23±140µm). Conclusion We developed a deep learning algorithm for automated analysis of intracoronary OCT images, that was able to accurately detect lipid plaques. Moreover, the algorithm showed promising results in terms of plaque characterization. With further refinements, the developed algorithm has the potential to reduce the time required for OCT interpretation, ultimately enabling more efficient, real-time use of this imaging modality in clinical practice.

12 The Development of a Pediatric Metamemory Questionnaire and Scoring Procedure

Objective:To create a standardized scoring procedure to evaluate open-ended responses as part of a novel questionnaire (Measure of Metamemory; MoM) designed to assess declarative metamemory in youth. Metamemory is an aspect of metacognition that is one’s knowledge of the factors related to storage and retrieval of information (Flavell 1971; Kreutzer et al., 1975), and includes both declarative metamemory (i.e., one’s knowledge about factors influencing memory) and procedural metamemory (i.e., one’s understanding of their own memory performance).Participants and Methods:Fourteen short vignettes related to memory were administered to 100 participants (age 6-12) with questions such as, “Two children hear a story they must remember. The first person is 5 years old. The second person is 12 years old. Who is most likely to remember it best?” After answering, they were then prompted to explain their answer (i.e., “Why?”) and their responses were recorded verbatim. To develop standardized and objective criteria for each of the 14 open-ended responses, responses from a subsample of 20 youth were collectively examined by the study team and a scoring structure similar to open-ended items on common intelligence tests (e.g., WISC-V/WAIS-IV) was created. Two points (full credit) were awarded for complete and thorough understanding of memory processes related to the question; 1 point was given for partially accurate or incomplete understanding of the related memory process; and 0 points for an inability to correctly express an understanding of relevant memory concepts. This scoring guide was then applied independently by each of the six raters to an additional 25 participants (ages 6-12 mean age (SD)). To assess the interrater reliability of this 3-point ordinal scoring system, we examined both Fleiss’ kappa and 2-way random-effects, single-rater, absolute agreement Intra-Class Correlations (ICC).Results:Across the six independent raters, reliability coefficients for each of the 14 items ranged from (Fleiss') k =. 277 to .792 (ICC ranged from .481 to .880). Of these 14 items, the kappa value was classified (using interpretation rules for Cohen’s kappa) as “substantial” for 10 items, “moderate” for 3 items, and “fair” for 1 item. Based on these lower inter-rater reliabilities, two items were subsequently removed from the measure to create the 12-item open-ended measure of metamemory, the MoM-12, with reliable scoring for youth as young as 6 years old.Conclusions:A consensus process established a quantifiable scoring procedure to assess open-ended responses related to youth’s knowledge of memory (e.g., metamemory). Reliability metrics identified acceptable interrater reliabilities in 12 of the 14 original items. Further examination of psychometric properties, including internal consistency and lines of evidence for validity is needed. The successful crafting of a scoring procedure is a first step towards developing a psychometrically sound measure (the MoM) to evaluate and study metamemory concepts objectively and reliably in youth.

Development and Validation of a Formative Assessment Tool for Nephrology Fellows' Clinical Reasoning.

Diagnostic errors are commonly driven by failures in clinical reasoning. Deficits in clinical reasoning are common among graduate medical learners, including nephrology fellows. We created and validated an instrument to assess clinical reasoning in a national cohort of nephrology fellows and established performance thresholds for remedial coaching. Experts in nephrology education and clinical reasoning remediation designed an instrument to measure clinical reasoning through a written patient encounter note from a web-based, simulated AKI consult. The instrument measured clinical reasoning in three domains: problem representation, differential diagnosis with justification, and diagnostic plan with justification. Inter-rater reliability was established in a pilot cohort ( n =7 raters) of first-year nephrology fellows using a two-way random effects agreement intraclass correlation coefficient model. The instrument was then administered to a larger cohort of first-year fellows to establish performance standards for coaching using the Hofstee method ( n =6 raters). In the pilot cohort, there were 15 fellows from four training program, and in the study cohort, there were 61 fellows from 20 training programs. The intraclass correlation coefficients for problem representation, differential diagnosis, and diagnostic plan were 0.90, 0.70, and 0.50, respectively. Passing thresholds (% total points) in problem representation, differential diagnosis, and diagnostic plan were 59%, 57%, and 62%, respectively. Fifty-nine percent ( n =36) met the threshold for remedial coaching in at least one domain. We provide validity evidence for a simulated AKI consult for formative assessment of clinical reasoning in nephrology fellows. Most fellows met criteria for coaching in at least one of three reasoning domains, demonstrating a need for learner assessment and instruction in clinical reasoning.

IODA-shoulder®, a 3-dimensional automated software for glenoid bone loss quantification in shoulder instability. A validation study

BackgroundGlenoid bone loss (GBL) calculation in patients with shoulder instability has a wide variability in methods and their reliability. The purpose of this study was to describe and validate a new semi-automated software developed to improve GBL calculation using a 3D imaging modeling (IODA-shoulder) and to compare the method to the PICO area method. Patients and methodsA semi-automated software to assess GBL was preliminarily developed and validated on 7 fresh frozen specimens (scapulae with artificially created glenoid defect), using water displacement method. Afterwards, the software was retrospectively used on CT images of 20 patients affected by recurrent shoulder dislocation. Inclusion criteria were: unilateral dislocation, minimum 2 dislocation episodes. Exclusion criteria were bone reconstruction of the glenoid, failed shoulder stabilization, bilateral dislocation, shoulder arthritis. Three-dimensional computed tomography images of bilateral shoulder were retrieved for each patient. Two methods to determine GBL were compared. The PICO surface area method and the new IODA method. We assessed the intra- and inter-rater reliability of the two methods with the Intraclass Correlation Coefficient (ICC), the Bland-Alman analysis, and Lin's concordance correlation coefficient (CCC). ResultsWe did not find a statistically significant difference between the mean volumes calculated with PICO and IODA methods, respectively 914 vs. 815 mm3, p=0.155. The analysis carried out by using the traditional PICO method showed a lower concordance rate among four observers than the higher concordance found using IODA method, regardless of the size and the location of the defect. The ICC agreement with PICO was significantly lower than with IODA (0.76 vs. 0.97). We found a poor CCC with PICO (from 0.65 to 0.81) and a substantial one with IODA (from 0.96 to 0.98). DiscussionThe intra- and inter-rater reliability using IODA method is significantly better than PICO method. The assessment of GBL using IODA method is time saving, avoiding significant inter- and intra-observer variation, mainly due to individual skill and experience in the method. Level of evidenceIV, experimental study.

A Photo Score For Aesthetic Outcome In Sagittal Synostosis: An ERN CRANIO Collaboration.

European Reference Network (ERN) CRANIO is focused on optimizing care for patients with rare or complex craniofacial anomalies, including craniosynostosis and/or rare ear, nose, and throat disorders. The main goal of ERN CRANIO is to collect uniform data on treatment outcomes for multicenter comparison. We aimed to develop a reproducible and reliable suture-specific photo score that can be used for cross-center comparison of phenotypical severity of sagittal synostosis and aesthetic outcome of treatment. We conducted a retrospective study among nonsyndromic sagittal synostosis patients aged <19 years. We included preoperative and postoperative photo sets from 6 ERN CRANIO centers. Photo sets included bird's eye, lateral, and anterior-posterior views. The sagittal synostosis photo score was discussed in the working group, and consensus was obtained on its contents. Interrater agreement was assessed with weighted Fleiss' Kappa and intraclass correlation coefficients.The photo score consisted of frontal bossing, elongated skull, biparietal narrowness, temporal hollowing, vertex line depression, occipital bullet, and overall phenotype. Each item was scored as normal, mild, moderate, or severe. Results from 36 scaphocephaly patients scored by 20 raters showed kappa values ranging from 0.38 [95% bootstrap CI: 0.31, 0.45] for biparietal narrowness to 0.56 [95% bootstrap CI: 0.47, 0.64] for frontal bossing. Agreement was highest for the sum score of individual items [intraclass correlation coefficients agreement 0.69 [95% CI: 0.57, 0.82]. This is the first large-scale multicenter study in which experts investigated a photo score to assess the severity of sagittal synostosis phenotypical characteristics. Agreement on phenotypical characteristics was suboptimal (fair-moderate agreement) and highest for the summed score of individual photo score items (substantial agreement), indicating that although experts interpret phenotypical characteristics differently, there is consensus on overall phenotypical severity.

Assessing the Smallest Detectable Change of the Kessler Psychological Distress Scale Score in an Adult Population in Japan.

Psychological distress is prevalent worldwide and can lead to the development of mental conditions such as major depression and anxiety disorders. It is essential to assess the severity of patient-reported outcomes to provide effective treatment for psychological distress. The Kessler Psychological Distress Scale (K6) is one of the most widely used psychological distress scales. However, the smallest detectable change (SDC) of the K6 score has not been elucidated. Therefore, the current study aimed to determine the SDC of the K6 score in a Japanese adult population. Participants aged 20-59 years who are native Japanese speakers were recruited from the panel list of a web research firm. The K6 score was assessed at baseline (T1) and at 2-week follow-up (T2). SDCs were calculated at the individual (SDCind) and group (SDCgroup) levels. Intraclass correlation coefficient agreement (ICCagreement) was calculated to assess test-retest reliability and Cronbach's alpha to evaluate internal consistency. A total of 3254 (1627 [50%] female) responded at T1 and T2. The mean (standard deviation) K6 scores were 5.71 (5.84) at T1 and 5.65 (5.83) at T2. The SDCind and SDCgroup of the K6 score were 8.47 (35.31%) and 0.15 (0.63%), respectively. The ICCagreement was 0.73, and the Cronbach's alpha was 0.94. Our study provided evidence on the reliability and interpretation of the K6 score. Calculating the SDC of the K6 score can help identify the significance of changes in psychological distress over time and can determine the efficacy of interventions for psychological distress.

ChatGPT in glioma adjuvant therapy decision making: ready to assume the role of a doctor in the tumour board?

ObjectiveTo evaluate ChatGPT‘s performance in brain glioma adjuvant therapy decision-making.MethodsWe randomly selected 10 patients with brain gliomas discussed at our institution’s central nervous system tumour board (CNS TB). Patients’ clinical...

Brassiere Cup Size Agreement between Patients and Plastic Surgeons: Do Surgeons and Patients Speak the Same Size Language?

Three-dimensional (3D) scans of 32 subjects were evaluated by 10 plastic surgeons estimating cup size using the American brassiere system. The surgeons were blinded to all parameters, including the 3D surface software-derived volume measures of the Vectra scan. The 3D scans of the anterior torsos were viewed. The plastic surgeons' estimations were compared with the cup sizes stated by the subjects (disclosed cup size), using simple and weighted Kappa statistics. Agreement between the estimated and disclosed brassiere sizes was only slight (0.1479 ± 0.0605) using a simple Kappa analysis. Even when a Fleiss-Cohen-weighted comparison was used, only moderate agreement (0.6231 ± 0.0589) was found. The interrater agreement intraclass correlation coefficient was 0.705. Rater accuracy varied. The percentage of time spent in cosmetic practice and gender were not significantly correlated with accuracy. Agreement between cup size disclosed by subjects and estimates by plastic surgeons was low. A miscommunication between the surgeon and patient may occur when using brassiere sizes to communicate wishes and estimates in procedures that involve changes in breast volume.

109 Use of the Horse Grimace Scale to evaluate markers of stress in horses involved in equine assisted activities

Much like humans, horses possess the ability to generate a wide range of facial behaviors. These expressions are a form of communication used to convey substantial amounts of information to other members of the herd, including stress. A variety of studies have developed ethograms to quantify the degree of facial behaviors expressed as a means of assessing the welfare of the horse. This is particularly important in working horses, such as those involved in equine assisted activities (EAA). The objective of this study was to use the Horse Grimace Scale (HGS) to assess the stress response of horses involved in EAA by numerically scoring facial muscular tension. The HGS is a scale that focuses on 6 specific facial action units to evaluate the degree of pain and distress experienced by a horse. Each unit was given a score of 0 (not present), 1 (moderately present), or 2 (obviously present). The highest possible score was 12, indicating the greatest degree of pain. Horses trained in EAA (EAA, n = 6) were paired with a veteran with posttraumatic stress disorder (PTSD) for a 30-min weekly lesson or assigned as astanding non-EAA trained control (CON, n = 6) for the duration of 8 wks. Images of the profile of the horses were taken at 3 time points during the lesson: 10 min, 20 min, and 30 min after the start of the session. After the sessions, images were organized and redacted if necessary and scored by 4 independent raters who had received training on the use of the HGS. Score data were analyzed using a negative binomial mixed model, repeated measures ANOVA with fixed effects of treatment (EAA vs CON), session, and time point, and their interactions and a random effect of horse. Intraclass correlation coefficient for inter-rater agreement was 0.446 (P = 2.17e-59). Statistically different means were separated using Tukey's method. Significance was set at P ≤ 0.05. Scores were higher (P = 0.0178) in wk 4 (5.2 ± 0.4, mean ± SE) than wk 1 (3.7 ± 0.4) among CON horses. No other significant differences were found. Participation in EAA did not appear to have an adverse effect on horses as defined by pain and distress scores on the HGS.