Average Intraclass Correlation Research Articles

METAVISION: a novel system for quantifying metamorphopsia in patients with myopic traction maculopathy

PurposeTo develop a novel system for quantifying metamorphopsia in patients with myopic traction maculopathy (MTM) and to explore the metamorphopsia pattern of MTM.DesignObservational, cross-sectional study.MethodsWe designed a new system.ResultsOf the...

Patient-specific temperature distribution prediction in laser interstitial thermal therapy: single-irradiation data-driven method

Laser interstitial thermal therapy (LITT) is popular for treating brain tumours and epilepsy. The strict control of tissue thermal damage extent is crucial for LITT. Temperature prediction is useful for predicting thermal damage extent. Accurately predicting in vivo brain tissue temperature is challenging due to the temperature dependence and the individual variations in tissue properties. Considering these factors is essential for improving the temperature prediction accuracy. Objective. To present a method for predicting patient-specific tissue temperature distribution within a target lesion area in the brain during LITT. Approach. A magnetic resonance temperature imaging (MRTI) data-driven estimation model was constructed and combined with a modified Pennes bioheat transfer equation (PBHE) to predict patient-specific temperature distribution. In the PBHE for temperature prediction, the individual specificity and temperature dependence of thermal tissue properties and blood perfusion, as well as the individual specificity of optical tissue properties were considered. Only MRTI data during one laser irradiation were required in the method. This enables the prediction of patient-specific temperature distribution and the resulting thermal damage region for subsequent ablations. Main results. Patient-specific temperature prediction was evaluated based on clinical data acquired during LITT in the brain, using intraoperative MRTI data as the reference standard. Our method significantly improved the prediction performance of temperature distribution and thermal damage region. The average root mean square error was decreased by 69.54%, the average intraclass correlation coefficient was increased by 37.5%, the average Dice similarity coefficient was increased by 43.14% for thermal damage region prediction. Significance. The proposed method can predict temperature distribution and thermal damage region at an individual patient level during LITT, providing a promising approach to assist in patient-specific treatment planning for LITT in the brain.

Exploring Radiomics Features Based on H&E Images as Potential Biomarkers for Evaluating Muscle Atrophy: A Preliminary Study.

Radiomics features have been widely used as novel biomarkers in the diagnosis of various diseases, but whether radiomics features derived from hematoxylin and eosin (H&E) images can evaluate muscle atrophy has not been studied. Therefore, this study aims to establish a new biomarker based on H&E images using radiomics methods to quantitatively analyze H&E images, which is crucial for improving the accuracy of muscle atrophy assessment. Firstly, a weightless muscle atrophy model was established by laying macaques in bed, and H&E images of the shank muscle fibers of the control and bed rest (BR) macaques were collected. Muscle fibers were accurately segmented by designing a semi-supervised segmentation framework based on contrastive learning. Then, 77 radiomics features were extracted from the segmented muscle fibers, and a stable subset of features was selected through the LASSO method. Finally, the correlation between radiomics features and muscle atrophy was analyzed using a support vector machine (SVM) classifier. The semi-supervised segmentation results show that the proposed method had an average Spearman's and intra-class correlation coefficient (ICC) of 88% and 86% compared to manually extracted features, respectively. Radiomics analysis showed that the AUC of the muscle atrophy evaluation model based on H&E images was 96.87%. For individual features, GLSZM_SZE outperformed other features in terms of AUC (91.5%) and ACC (84.4%). In summary, the feature extraction based on the semi-supervised segmentation method is feasible and reliable for subsequent radiomics research. Texture features have greater advantages in evaluating muscle atrophy compared to other features. This study provides important biomarkers for accurate diagnosis of muscle atrophy.

An artificial intelligence-driven scoring system to measure histological disease activity in ulcerative colitis.

Assessment and scoring of histological images in Ulcerative colitis (UC) is prone to inter- and intra-observer variability. This study aimed to investigate whether an artificial intelligence (AI) system developed using image processing and machine learning algorithms could measure histological disease activity based on the Nancy index. A total of 200 histological images of patients with UC were used in this study. A novel AI algorithm was developed using state-of-the-art image processing and machine learning algorithms based on deep learning and feature extraction. The cell regions of each image, followed by the Nancy index, were manually annotated and measured independently by four histopathologists. Manual and AI-automated measurements of the Nancy index score were conducted and assessed using the intraclass correlation coefficient (ICC). The 200-image dataset was divided into two groups (80% was used for training and 20% for testing). Intraclass correlation coefficient statistical analyses were performed to evaluate the AI tool and used as a reference to calculate the accuracy. The average ICC among the histopathologists was 89.3 and the average ICC between histopathologists and the AI tool was 87.2. The AI tool was found to be highly correlated with histopathologists. The high correlation of performance of the AI method suggests promising potential for inflammatory bowel disease clinical applications. A standardized automated histological AI-driven scoring system can potentially be used in daily inflammatory bowel disease practice to reduce training needs and resource use, eliminate the subjectivity of the pathologists, and assess disease severity for treatment decisions.

Reliability testing of the Health of the Nation Outcome Scales 2018.

The Health of the Nation Outcome Scales (HoNOS) is a widely used clinical measure designed to rate and monitor the outcomes of service users accessing specialist mental healthcare. Since its development (in 1996), numerous research studies have confirmed the HoNOS captures the aspects of care that it purports to (validity), and that clinicians' ratings are consistent both over time, and between different raters (reliability). In 2018, the HoNOS was reviewed with updates made to some terminology and other revisions intended to remove ambiguity in the guidance for raters. However, although the new version (HoNOS 2018) was accompanied by a recommendation that its validity and reliability be re-tested this was not undertaken. To our knowledge, this is the first study to re-assess the updated tool's reliability by measuring the level of agreement between different raters. Our findings confirm that there is an acceptable level of consistency between student mental health nurses that have been trained to use the (new) HoNOS 2018. The HoNOS is nationally mandated for use by all specialist mental healthcare providers in the UK. Our findings provide some assurance that, with appropriate update training and monitoring of organisational-level data sets, the original HoNOS glossary can safely be replaced with the HoNOS 2018 to ensure more contemporary routine outcome measurement can occur. INTRODUCTION: The Health of the Nation Outcome Scales (HoNOS) is a well-established clinician rated outcome measure for use in mental health services. Following an international review, an updated version (HoNOS 2018) was published with a recommendation that its psychometric properties be re-tested prior to widespread implementation. To date, only one such study has been published. To test the inter-rater agreement levels for HoNOS 2018. Third-year student mental health nurses received training to complete the HoNOS 2018. Following this timetabled session, they were each invited to independently rate two, randomly selected, videos of (simulated) patient interviews. The resulting data were then analysed to calculate the tool's internal consistency and inter-rater agreement levels. The 55 participants provided 106 ratings from four vignettes. Cronbach's alphas and McDonalds omegas confirmed the revised tool's internal consistency was acceptable. Average measure intraclass correlation coefficients for the four patient vignettes indicated excellent reliability. This study provides initial assurance that the HoNOS 2018 is a reliable clinician rated outcome measure suitable for use in routine clinical practice by relatively inexperienced mental health practitioners with limited training.

Diffusion tensor imaging within the healthy cervical spinal cord: Within- participants reliability and measurement error

BackgroundDiffusion tensor imaging (DTI) is a promising technique for the visualization of the cervical spinal cord (CSC) in vivo. It provides information about the tissue structure of axonal white matter, and it is thought to be more sensitive than other MR imaging techniques for the evaluation of damage to tracts in the spinal cord. AimThe purpose of this study was to determine the within-participants reliability and error magnitude of measurements of DTI metrics in healthy human CSC. MethodsA total of twenty healthy controls (10 male, mean age: 33.9 ± 3.5 years, 10 females, mean age: 47.5 ± 14.4 years), with no family history of any neurological disorders or a contraindication to MRI scanning were recruited over a period of two months. Each participant was scanned twice with an MRI 3 T scanner using standard DTI sequences. Spinal Cord Toolbox (SCT) software was used for image post-processing. Data were first corrected for motion artefact, then segmented, registered to a template, and then the DTI metrics were computed. The within-participants coefficients of variation (CV%), the single and average within-participants intraclass correlation coefficients (ICC) and Bland-Altman plots for WM, VC, DC and LC fractional anisotropy (FA), mean diffusivity (MD), axial diffusivity (AD), and radial diffusivity (RD) were determined for the cervical spinal cord (between the 2nd and 5th cervical vertebrae). ResultsDTI metrics showed poor to excellent within-participants reliability for both single and average ICC and moderate to high reproducibility for CV%, all variation dependent on the location of the ROI. The BA plots showed good within-participants agreement between the scan-rescan values. ConclusionResults from this reliability study demonstrate that clinical trials using the DTI technique are feasible and that DTI, in particular regions of the cord is suitable for use for the monitoring of degenerative WM changes.

Deep learning–based automatic segmentation of meningioma from T1-weighted contrast-enhanced MRI for preoperative meningioma differentiation using radiomic features

BackgroundThis study aimed to establish a dedicated deep-learning model (DLM) on routine magnetic resonance imaging (MRI) data to investigate DLM performance in automated detection and segmentation of meningiomas in comparison to manual segmentations. Another purpose of our work was to develop a radiomics model based on the radiomics features extracted from automatic segmentation to differentiate low- and high-grade meningiomas before surgery.MaterialsA total of 326 patients with pathologically confirmed meningiomas were enrolled. Samples were randomly split with a 6:2:2 ratio to the training set, validation set, and test set. Volumetric regions of interest (VOIs) were manually drawn on each slice using the ITK-SNAP software. An automatic segmentation model based on SegResNet was developed for the meningioma segmentation. Segmentation performance was evaluated by dice coefficient and 95% Hausdorff distance. Intra class correlation (ICC) analysis was applied to assess the agreement between radiomic features from manual and automatic segmentations. Radiomics features derived from automatic segmentation were extracted by pyradiomics. After feature selection, a model for meningiomas grading was built.ResultsThe DLM detected meningiomas in all cases. For automatic segmentation, the mean dice coefficient and 95% Hausdorff distance were 0.881 (95% CI: 0.851–0.981) and 2.016 (95% CI:1.439–3.158) in the test set, respectively. Features extracted on manual and automatic segmentation are comparable: the average ICC value was 0.804 (range, 0.636–0.933). Features extracted on manual and automatic segmentation are comparable: the average ICC value was 0.804 (range, 0.636–0.933). For meningioma classification, the radiomics model based on automatic segmentation performed well in grading meningiomas, yielding a sensitivity, specificity, accuracy, and area under the curve (AUC) of 0.778 (95% CI: 0.701–0.856), 0.860 (95% CI: 0.722–0.908), 0.848 (95% CI: 0.715–0.903) and 0.842 (95% CI: 0.807–0.895) in the test set, respectively.ConclusionsThe DLM yielded favorable automated detection and segmentation of meningioma and can help deploy radiomics for preoperative meningioma differentiation in clinical practice.

Validity and reliability of Arabic version of pediatric migraine disability assessment scale (Child Self-Report versus Parent Proxy-Report): a multi-center study

BackgroundPediatric Migraine Disability Assessment (PedMIDAS) is one of the most frequently used questionnaires to assess disability from migraine in pediatric patients. This work aimed to evaluate the validity and test–retest reliability of the Arabic version of the child self-report versus the parent proxy report PedMIDAS. We also aimed to test the agreement between children's and parents' reports of the scale.MethodsPedMIDAS was subjected to translation and back-translation, then applied to 112 pediatric patients fulfilling the migraine diagnostic criteria. This cross-sectional study was conducted on two visits, one week apart. At visit 1, the following data were obtained from the included pediatric patients: disease duration, migraine type, current treatment regimen, monthly migraine days (MMD) during the last month preceding the enrollment, and migraine intensity using the visual analogue scale. Then, each child and his parent were independently asked to fill out PedMIDAS and Child Self-Report of the Pediatric Quality of Life Inventory™ 4.0 (PedsQL™) to test the convergent validity of PedMIDAS. At visit 2, each child was requested to complete PedMIDAS again, and so was the parent to evaluate test–retest reliability.ResultsCronbach’s alpha was estimated to be 0.94 for each instrument. For the child-self report PedMIDAS, the average measure intraclass correlation coefficient (ICC) value was 0.992 (95%CI = 0.989–0.995), while it was estimated to be 0.990 for the parent-proxy report with 95%CI = 0.985–0.993, indicating excellent test–retest reliability for both instruments. The child-self report and the parent-proxy report PedMIDAS scores were significantly correlated with MMD, VAS, and all domains of the corresponding PedsQL, supporting convergent validity for both instruments. Agreement between parent and child on disability grading categories of PedMIDAS was substantial (κ = 0.644).ConclusionThe Arabic version of PedMIDAS was a valid and reliable instrument to assess disability from migraine in Arabic-speaking pediatric patients with migraine. Parent reports can be valuable as a complement to child reports for a comprehensive assessment of migraine.

Interobserver Reliability in the Classification of Thoracolumbar Fractures Using the AO Spine TL Injury Classification System Among 22 Clinical Experts in Spine Trauma Care.

Reliability study utilizing 183 injury CT scans by 22 spine trauma experts with assessment of radiographic features, classification of injuries and treatment recommendations. To assess the reliability of the AOSpine TL Injury Classification System (TLICS) including the categories within the classification and the M1 modifier. Kappa and Intraclass correlation coefficients were produced. Associations of various imaging characteristics (comminution, PLC status) and treatment recommendations were analyzed through regression analysis. Multivariable logistic regression modeling was used for making predictive algorithms. Reliability of the AO Spine TLICS at differentiating A3 and A4 injuries (N = 71) (K = .466; 95% CI .458 - .474; P < .001) demonstrated moderate agreement. Similarly, the average intraclass correlation coefficient (ICC) amongst A3 and A4 injuries was excellent (ICC = .934; 95% CI .919 - .947; P < .001) and the ICC between individual measures was moderate (ICC = .403; 95% CI .351 - .461; P < .001). The overall agreement on the utilization of the M1 modifier amongst A3 and A4 injuries was fair (K = .161; 95% CI .151 - .171; P < .001). The ICC for PLC status in A3 and A4 injuries averaged across all measures was excellent (ICC = .936; 95% CI .922 - .949; P < .001). The M1 modifier suggests respondents are nearly 40% more confident that the PLC is injured amongst all injuries. The M1 modifier was employed at a higher frequency as injuries were classified higher in the classification system. The reliability of surgeons differentiating between A3 and A4 injuries in the AOSpine TLICS is substantial and the utilization of the M1 modifier occurs more frequently with higher grades in the system.

P467 Towards AI-Augmented Clinical Decision Making: An Examination of ChatGPT's Utility in Acute Ulcerative Colitis Presentations

Abstract Background ChatGPT is a large language model based chatbot created by OpenAI. Since its release, ChatGPT has gained widespread attention among the healthcare community regarding its potential utility as a medical practice tool. In addition, ChatGPT can be adapted to serve as a clinical-decision support tool. In this study we explored the potential of ChatGPT as a decision support tool for acute Ulcerative Colitis (UC) presentations in the setting of the emergency department (ED). Methods Our investigation centered around 20 distinct acute UC presentations to the ED, accumulated over two years. Case summaries - embodying crucial data points such as symptoms, vital signs, and laboratory results - were processed by ChatGPT. For each case, we asked ChatGPT to assess disease severity based on the TrueLove and Witts classification, substituting erythrocyte sedimentation rate ≥30 with C-Reactive protein ≥12. Furthermore, it was to recommend hospitalization or outpatient care for each case based on the disease severity. The answers were compared with assessments made by our department's gastroenterologists and the actual decision made by the physician in the ED. Results Overall, ChatGPT categorized 12, 7 and 1 patient with severe, moderate and mild disease, respectively. For each case, ChatGPT supplied a detailed answer depicting severity of every variable of the criteria and an overall severity classification (table 1). Compared to our gastroenterologists’ assessments, ChatGPT graded 16/20 (80%) of the patients with the same severity. A high degree of reliability was found between the two assessments as the average measure intra-class correlation coefficient of absolute agreement was 0.839 (95% confidence interval 0.588-0.937, F= 5.95, p&amp;lt;0.001). Inconsistencies in four cases stemmed primarily from inaccurate cut-off values for systemic variables. Following severity assessment, ChatGPT leaned towards hospitalization for 16 out of 18 (88.9%) patients. For two moderate UC cases, however, it could not provide a decisive recommendation. Comparatively, only 12 out of the 20 patients were hospitalized in actual clinical practice. Conclusion In this unique study, findings suggest that Chat-GPT, has potential as a clinical decision-support tool in assessing UC severity and recommending suitable settings for further treatment. While this concept warrants further investigation and validation, its ability to evaluate a clinical scenario based on established criteria could greatly benefit the field of Inflammatory bowel disease and gastroenterology.

The development and validation of the S-scan-parental self-management support (S-scan - PS): A self-reflection tool for child healthcare professionals.

Professionals providing self-management support to parents regarding the care for their child with a chronic condition nowadays is an important aspect of child healthcare. This requires professionals to orient themselves towards partnership and collaboration with parents. The aims of the current study were the development and validation of the S-Scan-Parental self-management Support (S-scan - PS) as a tool for healthcare professionals to reflect on their attitude and practices regarding the support for parental self-management. An existing instrument was adapted together with field experts for professionals to self-evaluate their support for self-management of parents. The resulting 36-item self-report questionnaire was filled in by healthcare professionals in the Netherlands working with children and their parents. Cognitive interviews, exploratory and confirmatory factor analysis (CFA), and test-retest reliability analysis were part of the development and validation process. In total, 434 professionals, including physicians, physiotherapists, occupational therapists, and nurses, from 13 rehabilitation institutes and 5 medical centres participated. The cognitive interviews with child healthcare professionals indicated adequate face and content validity. The S-scan - PS scale had acceptable internal consistency (0.71 ≤ α ≤ 0.91) for the total score as well as the domain scores. CFA showed acceptable root mean square error of approximation (RMSEA) model fit (0.066), though not on other tested goodness-of-fit indices. Test-retest reliability of the instrument was moderate with an average intraclass correlation coefficient (ICC) = 0.61. The S-scan - PS fulfils important psychometric criteria for use by child healthcare professionals to reflect on parental self-management support. Such self-reflection might help to improve their approach towards supporting self-management of parents in the care for their child with a chronic condition. Further research is needed into the construct validity and test-retest reliability of the instrument.

PROJECT, PERSUADE, AND PROTECT: ADULT CHILDREN’S PREDICTION OF PARENTS’ HEALTH STATE VALUATIONS

Abstract Serious illness decision making for older adults frequently involves their adult children. Thus, it is important that adult children are familiar with parents’ illness preferences. In this mixed methods analysis (total N = 114), we examined concordance between older adults’ valuations of illness states, their children’s predictions of their valuations, reasoning behind children’s predictions, and children’s reactions to discordance between their predictions and parents’ valuations. Thirty-eight older parents (M age= 75.8) provided valuations for five different illness states. Two adult children (M age= 46.0) independently predicted their parent’s answers. Family triads participated in a semi-structured conversation about their responses. One-way random effects, absolute agreement, average measures intraclass correlation coefficients (ICCs) showed that average concordance within families was poor for incontinence (ICC = .172, 95% CI [-.497, .572]), dialysis (ICC = .300, 95% CI [-.264, .639]), tube feeding (ICC = .320, 95% CI [-.228, .649]), and chronic pain (ICC = .438, 95% CI [-.015, .710]), and moderate for Alzheimer disease (ICC = .556, 95% CI [.198,.771]). Results from qualitative thematic analysis indicated that children often projected their own values when attempting to predict parents’ valuations. When faced with discordance, many children attempted to persuade parents to change their valuations, while also making assertions to protect the parent’s autonomy. In summary, adult children navigating surrogate or shared decision making may face challenges inferring parent values versus their own and balancing their input with parent autonomy. Future research and interventions in late-life families could target these challenges.

A Critical Analysis of the Robustness of Radiomics to Variations in Segmentation Methods in 18F-PSMA-1007 PET Images of Patients Affected by Prostate Cancer.

Radiomics shows promising results in supporting the clinical decision process, and much effort has been put into its standardization, thus leading to the Imaging Biomarker Standardization Initiative (IBSI), that established how radiomics features should be computed. However, radiomics still lacks standardization and many factors, such as segmentation methods, limit study reproducibility and robustness. We investigated the impact that three different segmentation methods (manual, thresholding and region growing) have on radiomics features extracted from 18F-PSMA-1007 Positron Emission Tomography (PET) images of 78 patients (43 Low Risk, 35 High Risk). Segmentation was repeated for each patient, thus leading to three datasets of segmentations. Then, feature extraction was performed for each dataset, and 1781 features (107 original, 930 Laplacian of Gaussian (LoG) features, 744 wavelet features) were extracted. Feature robustness and reproducibility were assessed through the intra class correlation coefficient (ICC) to measure agreement between the three segmentation methods. To assess the impact that the three methods had on machine learning models, feature selection was performed through a hybrid descriptive-inferential method, and selected features were given as input to three classifiers, K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), Random Forest (RF), AdaBoost and Neural Networks (NN), whose performance in discriminating between low-risk and high-risk patients have been validated through 30 times repeated five-fold cross validation. Our study showed that segmentation methods influence radiomics features and that Shape features were the least reproducible (average ICC: 0.27), while GLCM features the most reproducible. Moreover, feature reproducibility changed depending on segmentation type, resulting in 51.18% of LoG features exhibiting excellent reproducibility (range average ICC: 0.68-0.87) and 47.85% of wavelet features exhibiting poor reproducibility that varied between wavelet sub-bands (range average ICC: 0.34-0.80) and resulted in the LLL band showing the highest average ICC (0.80). Finally, model performance showed that region growing led to the highest accuracy (74.49%), improved sensitivity (84.38%) and AUC (79.20%) in contrast with manual segmentation.

Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging.

We aimed to compare the performance and interobserver agreement of radiologists manually segmenting images or those assisted by automatic segmentation. We further aimed to reduce interobserver variability and improve the consistency of radiomics features. This retrospective study included 327 patients diagnosed with prostate cancer from September 2016 to June 2018; images from 228 patients were used for automatic segmentation construction, and images from the remaining 99 were used for testing. First, four radiologists with varying experience levels retrospectively segmented 99 axial prostate images manually using T2-weighted fat-suppressed magnetic resonance imaging. Automatic segmentation was performed after 2 weeks. The Pyradiomics software package v3.1.0 was used to extract the texture features. The Dice coefficient and intraclass correlation coefficient (ICC) were used to evaluate segmentation performance and the interobserver consistency of prostate radiomics. The Wilcoxon rank sum test was used to compare the paired samples, with the significance level set at p < 0.05. The Dice coefficient was used to accurately measure the spatial overlap of manually delineated images. In all the 99 prostate segmentation result columns, the manual and automatic segmentation results of the senior group were significantly better than those of the junior group (p < 0.05). Automatic segmentation was more consistent than manual segmentation (p < 0.05), and the average ICC reached >0.85. The automatic segmentation annotation performance of junior radiologists was similar to that of senior radiologists performing manual segmentation. The ICC of radiomics features increased to excellent consistency (0.925 [0.888~0.950]). Automatic segmentation annotation provided better results than manual segmentation by radiologists. Our findings indicate that automatic segmentation annotation helps reduce variability in the perception and interpretation between radiologists with different experience levels and ensures the stability of radiomics features.

Collecting evidence of validity for an assessment tool for Norwegian medical students’ non-technical skills (NorMS-NTS): usability and reliability when used by novice raters

BackgroundThe NorMS-NTS tool is an assessment tool for assessing Norwegian medical students’ non-technical skills (NTS). The NorMS-NTS was designed to provide student feedback, training evaluations, and skill-level comparisons among students at different study sites. Rather than requiring extensive rater training, the tool should capably suit the needs of busy doctors as near-peer educators. The aim of this study was to examine the usability and preliminary assess validity of the NorMS-NTS tool when used by novice raters.MethodsThis study focused on the usability of the assessment tool and its internal structure. Three raters used the NorMS-NTS tool to individually rate the team leader, a medical student, in 20 video-recorded multi-professional simulation-based team trainings. Based on these ratings, we examined the tools’ internal structure by calculating the intraclass correlation coefficient (ICC) (version 3.1) interrater reliability, internal consistency, and observability. After the rating process was completed, the raters answered a questionnaire about the tool’s usability.ResultsThe ICC agreement and the sum of the overall global scores for all raters were fair: ICC (3,1) = 0.53. The correlation coefficients for the pooled raters were in the range of 0.77–0.91. Cronbach’s alpha for elements, categories and global score were mostly above 0.90. The observability was high (95%-100%). All the raters found the tool easy to use, none of the elements were redundant, and the written instructions were helpful. The raters also found the tool easier to use once they had acclimated to it. All the raters stated that they could use the tool for both training and teaching.ConclusionsThe observed ICC agreement was 0.08 below the suggested ICC level for formative assessment (above 0.60). However, we know that the suggestion is based on the average ICC, which is always higher than a single-measure ICC. There are currently no suggested levels for single-measure ICC, but other validated NTS tools have single-measure ICC in the same range. We consider NorMS-NTS as a usable tool for formative assessment of Norwegian medical students’ non-technical skills during multi-professional team training by raters who are new to the tool. It is necessary to further examine validity and the consequences of the tool to fully validate it for formative assessments.

Inter-rater reliability of a novel objective endpoint for benign central airway stenosis interventions: Segmentation-based volume rendering of computed tomography scans.

To evaluate the reliability of a novel segmentation-based volume rendering approach for quantification of benign central airway obstruction (BCAO). A retrospective single-center cohort study. Data were ascertained using electronic health records at a tertiary academic medical center in the United States. Patients with airway stenosis located within the trachea on two-dimensional (2D) computed tomography (CT) imaging and documentation of suspected benign etiology were included. Four readers with varying expertise in quantifying tracheal stenosis severity were selected to manually segment each CT using a volume rendering approach with the available free tools in the medical imaging viewing software OsiriX (Bernex, Switzerland). Three expert thoracic radiologists were recruited to quantify the same CTs using traditional subjective methods on a continuous and categorical scale. The interrater reliability for continuous variables was calculated by the intraclass correlation coefficient (ICC) using a two-way mixed model with 95% confidence intervals (CI). Thirty-eight patients met the inclusion criteria, and fifty CT scans were selected for measurement. The most common etiology of BCAO was iatrogenic in 22 patients (58%). There was an even distribution of chest and neck CT imaging within our cohort. The average ICC across all four readers for the volume rendering approach was 0.88 (95% CI, 0.84 to 0.93), suggesting good to excellent agreement. The average ICC for thoracic radiologists for subjective methods on the continuous scale was 0.38 (95% CI, 0.20 to 0.55), suggesting poor to fair agreement. The kappa for the categorical approach was 0.26, suggesting a slight to fair agreement amongst the raters. In this retrospective cohort study, agreement was good to excellent for raters with varying expertise in airway cross-sectional imaging using a novel segmentation-based volume rendering approach to quantify BCAO. This proposed measurement outperformed our expert thoracic radiologists using conventional subjective grading methods.

Test-retest repeatability in macular retinal oximetry

ABSTRACT Clinical relevance Evaluation of retinal macular oxygen saturation in healthy controls can aid in understanding the pathological changes seen in similar locations of those with vascular diseases like diabetes. Background The aim of this study was to determine the test-retest repeatability of localised retinal oximetry measurements in the macula on the Zilia Oximeter within healthy subjects of different races, 18–40 years old. Oxygen saturation was measured between three time points within the same locations of the right eye. Methods Twenty seven subjects were included (aged 26.3 ± 3.6 years). All were confirmed to have healthy retinas and at least 6/9 vision. Oximetry measurements were taken using the Zilia to acquire local oxygen saturation measurements (300 µm diameter) at four points 3.1 degrees from the fovea in the superior/temporal, superior/nasal, inferior/temporal, and inferior/nasal locations. Oximetry measurements were taken twice on the same day 20 minutes apart and then again 1–2 weeks later. Oximetry data was analysed with intraclass correlation between visits. To assess intrasubject repeatability, the Bland-Altman repeatability coefficient and coefficient of variation were calculated. Results Average Intraclass correlation for the three acquisition times of the right eye was 0.78. The averaged intrasubject repeatability coefficient for the three acquisition times was 8.4. The averaged coefficient of variation was 5.4%. Conclusion The Zilia oximeter has good macular test-retest repeatability; however, multiple measurements may be needed to ensure accuracy.

Test-Retest Reliability of Functional Connectivity in Adolescents With Depression

BackgroundThe test-retest reliability of functional magnetic resonance imaging is critical to identifying reproducible biomarkers for psychiatric illness. Recent work has shown how reliability limits the observable effect size of brain-behavior associations, hindering detection of these effects. However, while a fast-growing literature has explored both univariate and multivariate reliability in healthy individuals, relatively few studies have explored reliability in populations with psychiatric illnesses or how this interacts with age. MethodsHere, we investigated functional connectivity reliability over the course of 1 year in a longitudinal cohort of 88 adolescents (age at baseline = 15.63 ± 1.29 years; 64 female) with major depressive disorder (MDD) and without MDD (healthy volunteers [HVs]). We compared a univariate metric, intraclass correlation coefficient, and 2 multivariate metrics, fingerprinting and discriminability. ResultsAdolescents with MDD had marginally higher mean intraclass correlation coefficient (μMDD = 0.34, 95% CI, 0.12–0.54; μHV = 0.27, 95% CI, 0.05–0.52), but both groups had poor average intraclass correlation coefficients (<0.4). Fingerprinting index was greater than chance and did not differ between groups (fingerprinting indexMDD = 0.75; fingerprinting indexHV = 0.91; Poisson tests p < .001). Discriminability indicated high multivariate reliability in both groups (discriminabilityMDD = 0.80; discriminabilityHV = 0.82; permutation tests p < .01). Neither univariate nor multivariate reliability was associated with symptom severity or edge-level effect size of group differences. ConclusionsOverall, we found little evidence for a relationship between depression and reliability of functional connectivity during adolescence. These findings suggest that biomarker identification in depression is not limited due to reliability compared with healthy samples and support the shift toward multivariate analysis for improved power and reliability.

Trimester-specific hemodynamics of per- and polyfluoroalkyl substances and its relation to lipid profile in pregnant women

Per- and polyfluoroalkyl substances (PFAS) are persistent organic pollutants widely detected in blood from pregnant women, yet limited study evaluated the trimester-specific variance of serum PFAS, and even less is known for the window of vulnerability to lipids metabolism interrupting effects of PFAS during pregnancy. We quantified 16 legacy, 3 emerging PFAS, and lipid biomarkers in 286 serum samples from 118 pregnant women. All target PFAS, except perfluorotridecanoic acid (PFTrDA), in maternal serum showed moderate to low temporal irregular variability across gestation (average intraclass correlation coefficients ≥ 0.57), while the generalized estimating equations showed a significant declining trend in the serum levels during pregnancy (p for trend < 0.05). The decline of 6:2 chlorinated polyfluorinated ether sulfonate in maternal serum was the greatest with a change of − 21.63% from 1st to 2nd trimester, which indicated a possible higher accumulation of this emerging PFAS in fetal compartment. Multiple linear regression, multiple informant model and Bayesian kernel machine regression showed a higher vulnerability in the 1st trimester to effects of PFAS exposure on serum lipids of pregnant women. The results highlighted the importance of the study timing of PFAS exposure during pregnancy.

Dynamic CT scanning of the knee: Combining weight bearing with real-time motion acquisition

BackgroundImaging the lower limb during weight-bearing conditions is essential to acquire advanced functional joint information. The horizontal bed position of CT systems however hinders this process. The purpose of this study was to validate and test a device to simulate realistic knee weight-bearing motion in a horizontal position during dynamic CT acquisition and process the acquired images. Methods“Orthostatic squats” was compared to “Horizontal squats” on a device with loads between 35% and 55% of the body weight (%BW) in 20 healthy volunteers. Intraclass Correlation Coefficient (ICC), and standard error of measurement (SEM), were computed as measures of the reliability of curve kinematic and surface EMG (sEMG) data. Afterwards, the device was tested during dynamic CT acquisitions on three healthy volunteers and three patients with patellofemoral pain syndrome. The respective images were processed to extract Tibial-Tuberosity Trochlear-Groove distance, Bisect Offset and Lateral Patellar Tilt metrics. ResultsFor sEMG, the highest average ICCs (SEM) of 0.80 (6.9), was found for the load corresponding to 42%BW. Kinematic analysis showed ICCs were the highest for loads of 42%BW during the eccentric phase (0.79–0.87) and from maximum flexion back to 20° (0.76). The device proved to be safe and reliable during the acquisition of dynamic CT images and the three metrics were computed, showing preliminary differences between healthy and pathological participants. ConclusionsThis device could simulate orthostatic squats in a horizontal position with good reliability. It also successfully provided dynamic CT scan images and kinematic parameters of healthy and pathological knees during weight-bearing movement.