Level Of Interobserver Agreement Research Articles

Interobserver Reproducibility of Ultrasound Attenuation Imaging Technology in Liver Fat Quantification.

This study aims to investigate the interobserver variability in the quantitative assessment of liver fat content using ultrasound attenuation imaging technology (USAT). This prospective, single-center study included 96 adult patients who were either diagnosed with or suspected of having metabolic dysfunction-associated steatotic liver disease. Independent observers, blinded to each other's assessments, evaluated hepatic steatosis visually and through USAT measurements. Separate measurements were taken at five intercostal and subcostal sites, and the median values of these measurements were recorded. The correlation between USAT measurements and visual steatosis grades was examined using Spearman's correlation test. Intraclass correlation coefficient (ICC) and Bland-Altman analysis were used to evaluate the interobserver variability of USAT measurements. Interobserver agreement for USAT measurements was excellent for the intercostal examination and good for the subcostal examination (p < 0.001). Body mass index did not significantly affect the level of interobserver agreement. Interobserver variability in Bland-Altman plots of USAT measurements was within the 95% limits of agreement. USAT measurements correlated very strongly with the visual degree of hepatic steatosis, both intercostal and subcostal (p < 0.001). USAT measurements were also significantly different between different visual degrees of hepatic steatosis (p < 0.001). In the assessment of hepatic steatosis, USAT measurements obtained from the intercostal space showed excellent agreement in terms of interobserver reproducibility.

A Deep Learning Application of Capsule Endoscopic Gastric Structure Recognition Based on a Transformer Model.

Gastric structure recognition systems have become increasingly necessary for the accurate diagnosis of gastric lesions in capsule endoscopy. Deep learning, especially using transformer models, has shown great potential in the recognition of gastrointestinal (GI) images according to self-attention. This study aims to establish an identification model of capsule endoscopy gastric structures to improve the clinical applicability of deep learning to endoscopic image recognition. A total of 3343 wireless capsule endoscopy videos collected at Nanfang Hospital between 2011 and 2021 were used for unsupervised pretraining, while 2433 were for training and 118 were for validation. Fifteen upper GI structures were selected for quantifying the examination quality. We also conducted a comparison of the classification performance between the artificial intelligence model and endoscopists by the accuracy, sensitivity, specificity, and positive and negative predictive values. The transformer-based AI model reached a relatively high level of diagnostic accuracy in gastric structure recognition. Regarding the performance of identifying 15 upper GI structures, the AI model achieved a macroaverage accuracy of 99.6% (95% CI: 99.5-99.7), a macroaverage sensitivity of 96.4% (95% CI: 95.3-97.5), and a macroaverage specificity of 99.8% (95% CI: 99.7-99.9) and achieved a high level of interobserver agreement with endoscopists. The transformer-based AI model can accurately evaluate the gastric structure information of capsule endoscopy with the same performance as that of endoscopists, which will provide tremendous help for doctors in making a diagnosis from a large number of images and improve the efficiency of examination.

Assessing the inter-observer and intra-observer reliability of radiographic measurements for size-specific dose estimates

BackgroundCalculating size-specific dose estimates (SSDEs) requires measurement of the patient’s anteroposterior (AP) and lateral thickness based on computed tomography (CT) images. However, these measurements can be subject to variation due to inter-observer and intra-observer differences. This study aimed to investigate the impact of these variations on the accuracy of the calculated SSDE.MethodsFour radiographers with 1–10 years of experience were invited to measure the AP and lateral thickness on 30 chest, abdomen, and pelvic CT images. The images were sourced from an internet-based database and anonymized for analysis. The observers were trained to perform the measurements using MicroDicom software and asked to repeat the measurements 1 week later. The study was approved by the institutional review board at Taibah University, and written informed consent was obtained from the observers. Statistical analyses were performed using Python libraries Pingouin (version 0.5.3), Seaborn (version 0.12.2), and Matplotlib (version 3.7.1).ResultsThe study revealed excellent inter-observer agreement for the calculated effective diameter and AP thickness measurements, with Intraclass correlation coefficients (ICC) values of 0.95 and 0.96, respectively. The agreement for lateral thickness measurements was lower, with an ICC value of 0.89. The second round of measurements yielded nearly the same levels of inter-observer agreement, with ICC values of 0.97 for the effective diameter, 1.0 for AP thickness, and 0.88 for lateral thickness. When the consistency of the observer was examined, excellent consistency was found for the calculated effective diameter, with ICC values ranging from 0.91 to 1.0 for all observers. This was observed despite the lower consistency in the lateral thickness measurements, which had ICC values ranging from 0.78 to 1.0.ConclusionsThe study’s findings suggest that the measurements required for calculating SSDEs are robust to inter-observer and intra-observer differences. This is important for the clinical use of SSDEs to set diagnostic reference levels for CT scans.

Interobserver and intraobserver agreement in PET/CT with [18F]DCFPyL according to TNM molecular and PSMA-RADS 2.0 criteria

PurposeThe aim of this study was to determine the agreement between three observers with different levels of experience using the PSMA-RADS 2.0 criteria and the miTNM system for the interpretation of PET-PSMA with [18F]DCFPyL in males with prostate cancer. Materials and MethodsPET-PSMA images from 114 prostate cancer patients were blindly reported twice by three different observers at intervals of 8 weeks. The evaluations were performed according to the molecular imaging TNM (miTNM) and PSMA-RADS 2.0 criteria. We used Fleiss’ Kappa to analyse inter and intraobserver agreements. ResultsModerate overall agreement was obtained in the assessment of the PET-PSMA results (Fleiss'k = 0.53; 95% CI 0.45−0.62; p < 0.001), with significant agreement in the miT, miN and miM reports. There was a substantial level of agreement in the reporting of prostatic disease and lymphatic involvement (Fleiss'k = 0.66 and 0.65), being lower than that observed in the reporting of metastatic disease (Fleiss'k = 0.86), especially in the M0 group (Fleiss'k = 0.99). Upon re-evaluation of the images, observer 1 had moderate overall agreement for miT (Fleiss'k = 0.51) and substantial agreement for miN and miM (Fleiss'k 0.75 and 0.63, respectively). ConclusionsThe use of a structured scoring system such as PSMA-RADS 2.0, as well as the miTNM classification system in the interpretation of PET-PSMA images in prostate cancer patients, provides a highly reproducible report format. High levels of interobserver and intraobserver agreement are found, especially when ruling out disease, which supports its use in routine clinical practice.

Quantifying Lumbar Foraminal Volumetric Dimensions: Normative Data and Implications for Stenosis-Part 2 of a Comprehensive Series.

Lumbar foraminal stenosis (LFS) occurs primarily due to degenerative changes in older adults, affecting the spinal foramina and leading to nerve compression. Characterized by pain, numbness, and muscle weakness, LFS arises from structural changes in discs, joints, and ligaments, further complicated by factors like inflammation and spondylolisthesis. Diagnosis combines patient history, physical examination, and imaging, while management ranges from conservative treatment to surgical intervention, underscoring the need for a tailored approach. This multicenter study, conducted over six years at a tertiary hospital, analyzed the volumetric dimensions of lumbar foramina and their correlation with nerve structures in 500 patients without lumbar pathology. Utilizing high-resolution MRI with a standardized imaging protocol, eight experienced researchers independently reviewed the images for accurate measurements. The study emphasized quality control through the calibration of measurement tools, double data entry, validation checks, and comprehensive training for researchers. To ensure reliability, interobserver and intraobserver agreements were analyzed, with statistical significance determined by kappa statistics and the Student's t-test. Efforts to minimize bias included blinding observers to patient information and employing broad inclusion criteria to mitigate referral and selection biases. The methodology and findings aim to enhance the understanding of normal lumbar foramina anatomy and its implications for diagnosing and treating lumbar conditions. The study's volumetric analysis of lumbar foramina in 500 patients showed a progressive increase in foraminal volume from the L1/L2 to the L5/S1 levels, with significant enlargement at L5/S1 indicating anatomical and biomechanical complexity in the lumbar spine. Lateral asymmetry suggested further exploration. High interobserver and intraobserver agreement levels (ICC values of 0.91 and 0.95, respectively) demonstrated the reliability and reproducibility of measurements. The patient cohort comprised 58% males and 42% females, highlighting a balanced gender distribution. These findings underscore the importance of understanding foraminal volume variations for lumbar spinal health and pathology. Our study significantly advances spinal research by quantifying lumbar foraminal volumes, revealing a clear increase from the L1/L2 to the L5/S1 levels, indicative of the spine's adaptation to biomechanical stresses. This provides clinicians with a precise tool to differentiate between pathological narrowing and normal variations, enhancing the detection and treatment of lumbar foraminal stenosis. Despite limitations like its cross-sectional design, the strong agreement in measurements underscores the method's reliability, encouraging future research to further explore these findings' clinical implications.

Forensic age estimation by MRI of the knee – comparison of two classifications for ossification stages in a German population

Aim and objectivesIn forensic age estimation e.g. for judicial proceedings surpassed age thresholds can be legally relevant. To examine age related differences in skeletal development the recommendations by the Study Group on Forensic Age Diagnostics (AGFAD) are based on ionizing radiation (among others orthopantomograms, plain x-rays of the hand). Vieth et al. and Ottow et al. proposed MRI-classifications for the epiphyseal-diaphyseal fusion of the knee joint to define different age groups in healthy volunteers. The aim of the present study was to directly compare these two classifications in a large German patient population.Materials and methodsMRI of the knee joint of 900 patients (405 female, 495 male) from 10 to 28 years of age were retrospectively analyzed. Acquired T1-weighted turbo spin-echo sequence (TSE) and T2-weighted sequence with fat suppression by turbo inversion recovery magnitude (TIRM) were analyzed for the two classifications. The different bony fusion stages of the two classifications were determined and the corresponding chronological ages assigned. Differences between the sexes were analyzed. Intra- and inter-observer agreements were determined using Cohen’s kappa.ResultsWith the classification of Ottow et al. it was possible to determine completion of the 18th and 21st year of life in both sexes. With the classification of Vieth et al. completion of the 18th year of life for female patients and the 14th and 21st year of life in both sexes could be determined. The intra- and inter-observer agreement levels were very good (κ > 0.82).ConclusionIn the large German patient cohort of this study it was possible to determine the 18th year of life with for both sexes with the classification of Ottow et al. and for female patients with the classification of Vieth et al. It was also possible to determine the 21st year of life for all bones with the classification of Ottow et al. and for the distal femur with the classification of Vieth et al.

Digital matching of palatal rugae patterns for forensic identification in edentulous denture wearers.

Identifying edentulous victims in forensic contexts poses a significant challenge. It has been reported that having a denture to reproduce and compare Palatal Rugae (PR) patterns is crucial for identifying edentulous individuals, yet there are no validated protocols for conducting this procedure. In this study, a new method was developed and validated for plaster molding of the internal surface of upper dentures, along with a protocol involving focus-stacked photographs and reference scales. Thirty-eight edentulous subjects participated in the study, obtaining plaster models of the patients (PM), and their dentures (AM) were standardized. The AM/PM model was highlighted and photographed with ten shots in different z-focuses for each model, using standard lighting and aligning the position of the PR perpendicular to the photographic axis using tubular spirit levels and modeling clay. The images were processed using the stacking technique and analyzed by three observers through the proposed protocol using Adobe® Photoshop®. The results were analyzed based on the intra-observer and inter-observer agreement levels, with a 95% confidence interval. This study demonstrated high-precision intra-observer and inter-observer agreement (k = 1) in the matching of Palatal Rugae (PR) and maxillary morphology obtained from participants and their dentures. The protocol is simple, cost-effective, and precise. It enables standardization of the technique for obtaining plaster models, and the exposure of PR and photographic protocol minimizes the presence of artifacts in the images, thereby reducing the likelihood of errors and promoting the reproducibility of both the recording technique and the comparison of the PR.

Adult age estimation from the sternum using maximum intensity projection images of CT and data mining in a Chinese population.

This study aimed to explore and develop data mining models for adult age estimation based on CT reconstruction images from the sternum. Maximum intensity projection (MIP) images of chest CT were retrospectively collected from a modern Chinese population, and data from 2700 patients (1349 males and 1351 females) aged 20 to 70years were obtained. A staging technique within four indicators was applied. Several data mining models were established, and mean absolute error (MAE) was the primary comparison parameter. The intraobserver and interobserver agreement levels were good. Within internal validation, the optimal data mining model obtained the lowest MAE of 9.08 in males and 10.41 in females. For the external validation (N = 200), MAEs were 7.09 in males and 7.15 in females. In conclusion, the accuracy of our model for adult age estimation was among similar studies. MIP images of the sternum could be a potential age indicator. However, it should be combined with other indicators since the accuracy level is still unsatisfactory.

Hypo-attenuating Berry Sign as a Novel Imaging Marker of Ruptured Aneurysm in Patients with Subarachnoid Hemorrhage; a Diagnostic Accuracy Study.

Aneurysmal subarachnoid hemorrhage (SAH) constitutes a life-threatening condition, and identifying the ruptured aneurysm is essential for further therapy. This study aimed to evaluate the diagnostic accuracy of hypo-attenuating berry sign (HBS) observed on computed tomography (CT) scan in distinguishing ruptured aneurysms. In this diagnostic accuracy study, patients who had SAH and underwent non-enhanced brain CT scan were recruited. The HBS was defined as a hypo-attenuating area with an identifiable border in the blood-filled hyper-dense subarachnoid space. The screening performance characteristics of HBS in identifying ruptured aneurysms were calculated considering the digital subtraction angiography (DSA) as the gold standard. A total of 129 aneurysms in 131 patients were analyzed. The overall sensitivity and specificity of HBS in the diagnosis of aneurysms were determined to be 78.7% (95%CI: 73.1% - 83.4%) and 70.7% (95%CI: 54.3% - 83.4%), respectively. Notably, the sensitivity increased to 90.9% (95%CI: 84.3% - 95.0%) for aneurysms larger than 5mm. The level of inter-observer agreement for assessing the presence of HBS was found to be substantial (kappa=0.734). The diagnostic accuracy of HBS in individuals exhibited enhanced specificity, sensitivity, and reliability when evaluating patients with a solitary aneurysm or assessing ruptured aneurysms. The multivariate logistic regression analysis revealed a statistically significant relationship between aneurysm size and the presence of HBS (odds ratios of 1.667 (95%CI: 1.238 - 2.244; p < 0.001) and 1.696 (95%CI: 1.231 - 2.335; p = 0.001) for reader 1 and reader 2, respectively). The HBS can serve as a simple and easy-to-use indicator for identifying a ruptured aneurysm and estimating its size in SAH patients. .

Effect of Axillary Radiograph Rotation and Medial Scapular Visualization on Glenoid Retroversion Measurement.

Axillary radiographs enable the measurement of glenoid retroversion, which is associated with worsened clinical outcomes and glenoid loosening following total shoulder arthroplasty. Due to the variability in radiographic technique, this study aims to determine if the accuracy of retroversion measured by axillary radiograph is affected by 1) scapular rotation and/or 2) proper visualization of the medial scapula. Using five cadaveric scapulae, investigators obtained axillary radiographs in true neutral position as well as in 10° and 20° of anterior and posterior rotation. For each radiograph, two fellowship trained shoulder surgeons measured glenoid retroversion with complete visualization of the scapula (Technique 1) and with visualization limited to the lateral half of scapula (Technique 2). The observers also measured glenoid retroversion by CT scan to use as a gold standard technique. Spearman's Rho was used to assess agreement between measurements. Average glenoid retroversion of the five scapulae assessed by CT scan was 3.8° (R: 1.5-6.9). Measurements obtained using Technique 1 demonstrated improved levels of interobserver agreement (ICC: 0.412) compared to measurements obtained with Technique 2, which demonstrated no agreement (ICC: 0.103). Scapular rotation was inconsistently associated with agreement using both techniques. The reliability of glenoid retroversion measurements was limited by incomplete visualization of the medial scapular spine. When measuring retroversion to the base of the scapular spine, improved agreement and accuracy were seen with various degrees of scapular rotation.

Comparison of T2-Weighted 3D-Space and T2-Weighted 2D-TSE Sequences in Lumbar Spine Magnetic Resonance Imaging (MRI)

Background: Spine-related issues like low back pain have substantially increased in the past few decades across all age categories having various reasons. Objectives: To compare the image quality and diagnostic value of 3D space MRI sequence with 2D TSE MRI sequence in lumbar spine imaging. Methods: Fifty patients, irrespective of gender having age between 30-70 years, who presented with lower back pain were enrolled for the study. Lumbar spine imaging was done using 2D TSE and 3D space sequences and the subsequent images were evaluated by two senior consultant radiologists for qualitative analysis of visibility and pathologic indices of both sequences. Data was analyzed on SPSS 25.0. Statistical analysis was done using Cohen's kappa (k). Results: Female participants were 56% of total study population. Mean age of study participants found to be 53.62±10.76 years. Statistically significant difference (p-value =0.000) was noted for visibility score of all the lumbar spine components, when the 3D SPACE and 2D TSE images were compared with 3D-SPACE images found being superior in visibility. Inter- observer agreement for visibility of various regions of lumber spine was nearly perfect (k&gt; 0.8) and substantial (0.6 to 0.8) for 3D-SPACE, while substantial (0.8&lt;k&gt; 0.6) for 2D-TSE. Pathologic indexes had a significant level of inter-observer agreement (k &gt; 0.6). Inter-observer agreement for 3D sequence (k = 0.72) was greater than that for 2D-TSE sequence (k = 0.64). Conclusion: Images from 3D-SPACE sequences provide superior visibility over 2D-TSE and could be recommended as alternative imaging method for lumbar spine pathologies. Higher inter observer agreement illustrate that it may be included in routine sequences to make the better diagnosis for lumbar spine pathologies correctly.

Subscapularis tendon tear detection using axial internal rotation MRI: semiquantitative and quantitative analysis.

Magnetic resonance image (MRI) of the subscapularis tendon plays an important role in preoperative planning. This retrospective study aimed to evaluate the diagnostic value and quantitative measurement of an additional internal rotation sequence in the detection of partial subscapularis tendon tears. The study included 76 patients who underwent arthroscopy and magnetic resonance (MR) shoulder between January 2018 to December 2019. Three different sets of images were evaluated in each case to determine the diagnostic value in the detection of partial subscapularis tendon tears including Set 1: standard axial fat-suppressed proton density (PD/FS) image and sagittal fat-suppressed T2 weight image (T2W/FS) images, Set 2: standard axial PD/FS and internal rotation PD/FS images, and Set 3: standard axial PD/FS, sagittal T2W/FS and axial internal rotation PD/FS images. Subscapularis tendon tear was diagnosed by arthroscopy and patients with or without tears were grouped. The coracohumoral distance (CHD), coracoglenoid angle (CGA), coracohumeral angle (CHA), CHD difference and CHD ratio were evaluated and compared between groups using univariate and multivariate analysis. The interreader agreement was assessed. The cut-off point for the prediction of subscapularis tears was calculated. Twenty-nine shoulders revealed partial subscapularis tendon tears (29/76, 38.2%). Imaging Set 3 provided the highest sensitivity and accuracy {79-83% [confidence interval (CI): 0.60-0.95], 75-76% (CI: 0.63-0.85)}, compared to image Set 2 [31-58% (CI: 0.15-0.76), 67-68% (CI: 0.55-0.79)] and Set 1 [17-21% (CI: 0.06-0.40], 61-66% (CI: 0.54-0.76)], and a moderate level of interobserver agreement (Kappa =0.55). Axial CHD [odd ratio (OR) =1.48, P=0.044], internal rotate CHD (OR =0.68, P=0.02), CHD difference (OR =2.58, P<0.001), and CHD ratio (OR =1.34, P<0.001) were associated with subscapularis tears. A CHD difference and CHD ratio of more than 0.04 mm and 1.01 achieved a 90% sensitivity and 72% specificity, both. Internal rotation during MRI can increase diagnostic accuracy for subscapularis tendon partial tears. The CHD differences and CHD ratio are useful parameters to indicate subscapularis tears. This technique may improve preoperative management and reduce the consequences of delayed diagnosis and treatment.

Prognostic role of magnetic resonance imaging of the abdomen with intravenous contrast and magnetic resonance cholangiopancreatography in primary sclerosing cholangitis.

To evaluate the usefulness of Anali scores, determined by magnetic resonance imaging, for predicting the prognosis of primary sclerosing cholangitis (PSC) and to analyze interobserver variability, as well as to assess the impact of periportal edema and heterogeneous signal intensity on diffusion-weighted imaging of the liver. This was a retrospective cohort study of 29 patients with PSC and baseline magnetic resonance imaging. Anali scores, without gadolinium (0-5 points) and with gadolinium (0-2 points), were calculated by two radiologists. Clinical end-points included liver transplantation, cirrhotic decompensation, and death. We calculated intraclass correlation coefficients (ICCs) for interobserver agreement on the Anali scores, performed Kaplan-Meier survival analysis comparing event-free survival among the score strata, and calculated the areas under receiver operating characteristic curves to determine sensitivity and specificity. Among the patients with a clinical event, the median Anali score was 4 (interquartile range [IQR], 2-5) without gadolinium and 2 (IQR, 1-2) with gadolinium, compared with 1 (IQR, 1.0-2.5) and 1 (IQR, 0.25-1.0), respectively, among those without a clinical event. The ICC was 0.79 (95% confidence interval: 0.57-0.91) for the Anali score with gadolinium and 0.99 (95% confidence interval: 0.98-0.99) for the Anali score without gadolinium. Periportal edema and heterogeneous signal intensity in the liver on diffusion-weighted imaging showed no statistical impact on clinical events (p = 0.65 and p = 0.5, respectively). Anali scores correlate with clinical events in PSC, with a high level of interobserver agreement.

CCTA-based CABG SYNTAX Score: a tool to evaluate completeness of coronary segment revascularization after bypass surgery

To describe the updated coronary computed tomographic angiography (CCTA)-based coronary artery bypass graft (CABG) anatomic SYNTAX Score (aSS) and assess its utility and reproducibility for assessing the completeness of revascularization after CABG. The CCTA-CABG aSS is a visual assessment using CCTA post-CABG which quantifies the failure in effectively grafting stenotic coronary segments, and therefore assesses the completeness of surgical revascularization. It is calculated by subtracting the aSS of successfully anastomosed coronary segments from the aSS of the native coronary tree. The inter-observer reproducibility of the CCTA-CABG aSS was evaluated in 45 consecutive patients with three-vessel disease with or without left main disease who underwent a CCTA 30 days (± 7 days) after CABG. The CCTA-CABG aSS was evaluated in 45 consecutive patients with 117 bypass grafts and 152 anastomoses. The median native coronary aSS was 35.0 [interquartile range (IQR) 27.0–41.0], whilst the median CCTA-CABG aSS was 13.0 (IQR 9.0–20.5). The inter-observer level of agreement for the native coronary aSS and the CCTA-CABG aSS were both substantial with respective Kappas of 0.67 and 0.61. The CCTA-CABG aSS was feasible in all patients who underwent CABG for complex coronary artery disease with substantial inter-observer reproducibility, and therefore can be used to quantify the completeness of revascularization after CABG.

Magnetic resonance imaging of the knee anteromedial quadrant

ObjectiveThis study aims to evaluate the possibility of characterizing an extra-articular thickening in the knee anteromedial quadrant in routine MRI scans.Materials and methodsFirstly, in a pilot study, for a better understanding of this extra-articular thickening trajectory in MRI, polytetrafluoroethylene (PTFE) tubes were attached to the ligament structure topography in two dissected pieces. Afterward, 100 knee MRI studies were randomly selected from our database, and 97 met the inclusion criteria. Two musculoskeletal radiologists interpreted the exams separately. Both had previously studied the ligament in the cadaveric knee MRI with the PTFE tube.ResultsThe intraobserver and interobserver agreement for the ligament identification was calculated using Cohen’s Kappa coefficient. The first radiologist identified the structure in 41 of the 97 scans (42.2%), and the second radiologist in 38 scans (39.2%). The interobserver agreement was substantial, with a Kappa of 0.68 and an agreement of 84.5%. The results suggest that this extra-articular thickening, recently called Anterior Oblique Ligament (AOL) in the literature, is a structure that can be frequently visualized on MRI scans with a high level of interobserver agreement in a relatively large number of exams.ConclusionTherefore, this study indicates that MRI is a promising method for evaluating this anteromedial thickening, and it may be used for future studies of the Anterior Oblique Ligament.

Reliability of whole mount radical prostatectomy histopathology as the ground truth for artificial intelligence assisted prostate imaging

The development of artificial intelligence–based imaging techniques for prostate cancer (PCa) detection and diagnosis requires a reliable ground truth, which is generally based on histopathology from radical prostatectomy specimens. This study proposes a comprehensive protocol for the annotation of prostatectomy pathology slides. To evaluate the reliability of the protocol, interobserver variability was assessed between five pathologists, who annotated ten radical prostatectomy specimens consisting of 74 whole mount pathology slides. Interobserver variability was assessed for both the localization and grading of PCa. The results indicate excellent overall agreement on the localization of PCa (Gleason pattern ≥ 3) and clinically significant PCa (Gleason pattern ≥ 4), with Dice similarity coefficients (DSC) of 0.91 and 0.88, respectively. On a per-slide level, agreement for primary and secondary Gleason pattern was almost perfect and substantial, with Fleiss Kappa of .819 (95% CI .659–.980) and .726 (95% CI .573–.878), respectively. Agreement on International Society of Urological Pathology Grade Group was evaluated for the index lesions and showed agreement in 70% of cases, with a mean DSC of 0.92 for all index lesions. These findings show that a standardized protocol for prostatectomy pathology annotation provides reliable data on PCa localization and grading, with relatively high levels of interobserver agreement. More complicated tissue characterization, such as the presence of cribriform growth and intraductal carcinoma, remains a source of interobserver variability and should be treated with care when used in ground truth datasets.

Left ventricular function evaluation of the fetal heart: reference intervals and inter-observer variability of 2D speckle-tracking echocardiography measurements.

Measurement of myocardial strain using 2D speckle-tracking echocardiography can successfully quantify ventricular function, being considered superior to conventional echocardiography. This study aimed to establish reference intervals, interobserver agreements and reliability of two fetal echocardiographic parameters which reflect left ventricular myocardial function, left ventricular apical 4 chamber endo peak strain (AP4pLS) and ejection fraction (EF). We conducted a prospective study on 103 healthy fetuses. In each case, cardiac ultrasound images obtained were stored and afterwards were subject to offline 2D speckle-tracking echocardiographic analyses. In 15 randomly chosen subjects a second examiner also carried out an offline analysis of the 4-chamber view and the archived images, in order to assess inter-observer reproducibility and agreement level. Our study group was sub-divided into four different gestational age groups. Reference ranges were established for the two measured parameters, AP4pLS and EF, which did not differ significantly between four different gestational age groups (p=0.98 and p=0.64) and neither correlated with gestational age progression (p=0.37 and p=0.08). An excellent level of agreement between the two examiners was found for the echocardiographic measurements, expressed through an intra-class correlation coefficient (ICC) value of 0.85 (0.62-0.94 for 95%CI) for AP4pLS and 0.78 (0.47- 0.92 for 95% CI) for EF. Speckle tracking AP4pLS and EF parameters are useful for assessment of ventricular myocardial function in healthy fetuses and can be reliably reproduced by two different skilled examiners. Further studies conducted on larger populations are required to standardize reference values of fetal speckle-tracking measurements.

Can artificial intelligence help decision-making in arthroscopy? Part 1: Use of a standardized analysis protocol improves inter-observer agreement of arthroscopic diagnostic assessments of the long head of biceps tendon in small rotator cuff tears

IntroductionInjuries of the long head of biceps (LHB) tendon are common but difficult to diagnose clinically or using imaging. Arthroscopy is the preferred means of diagnostic assessment of the LHB, but it often proves challenging. Its reliability and reproducibility have not yet been assessed. Artificial intelligence (AI) could assist in the arthroscopic analysis of the LHB. The main objective of this study was to evaluate the inter-observer agreement for the specific LHB assessment, according to an analysis protocol based on images of interest. The secondary objective was to define a video database, called “ground truth”, intended to create and train AI for the LHB assessment. HypothesisThe hypothesis was that the inter-observer agreement analysis, on standardized images, was strong enough to allow the “ground truth” videos to be used as an input database for an AI solution to be used in making arthroscopic LHB diagnoses. Materials and methodOne hundred and ninety-nine sets of standardized arthroscopic images of LHB exploration were evaluated by 3 independent observers. Each had to characterize the healthy or pathological state of the tendon, specifying the type of lesion: partial tear, hourglass hypertrophy, instability, fissure, superior labral anterior posterior lesion (SLAP 2), chondral print and pathological pulley without instability. Inter-observer agreement levels were measured using Cohen's Kappa (K) coefficient and Kappa Accuracy. ResultsThe strength of agreement was moderate to strong according to the observers (Kappa 0.54 to 0.7 and KappaAcc from 86 to 92%), when determining the healthy or pathological state of the LHB. When the tendon was pathological, the strength of agreement was moderate to strong when it came to a partial tear (Kappa 0.49 to 0.71 and KappaAcc from 85 to 92%), fissure (Kappa −0.5 to 0.7 and KappaAcc from 36 to 93%) or a SLAP tear (0.54 to 0.88 and KappaAcc from 90 to 97%). It was low for unstable lesion (Kappa 0.04 to 0.25 and KappaAcc from 36 to 88%). ConclusionThe analysis of the LHB, from arthroscopic images, had a high level of agreement for the diagnosis of its healthy or pathological nature. However, the agreement rate decreased for the diagnosis of rare or dynamic tendon lesions. Thus, AI engineered from human analysis would have the same difficulties if it was limited only to an arthroscopic analysis. The integration of clinical and paraclinical data is necessary to improve the arthroscopic diagnosis of LHB injuries. It also seems to be an essential prerequisite for making a so-called “ground truth” database for building a high-performance AI solution. Level of evidenceIII; inter-observer prospective series.

Interpretation of Cardiac Standstill in Children Using Point-of-Care Ultrasound

This study aimed to determine the level of agreement among pediatric emergency medicine (PEM) physicians in whether various point-of-care ultrasound (POCUS) video clips represent cardiac standstill in children and to highlight the factors that may be associated with the lack of agreement. A single, online, cross-sectional, convenience sample survey was administered to PEM attendings and fellows with variable ultrasound experience. PEM attendings with an experience of 25 cardiac POCUS scans or more were the primary subgroup based on ultrasound proficiency set by the American College of Emergency Physicians. The survey contained 11 unique, 6-second video clips of cardiac POCUS performed during pulseless arrest in pediatric patients and asked the respondent if the video clip represented a cardiac standstill. The level of interobserver agreement was determined using the Krippendorff's α (Kα) coefficient across the subgroups. A total of 263 PEM attendings and fellows completed the survey (9.9% response rate). Of the 263 total responses, 110 responses were from the primary subgroup of experienced PEM attendings with at least 25 previously seen cardiac POCUS scans. Across all video clips, PEM attendings with 25 scans or more had an acceptable agreement (Kα=0.740; 95% CI 0.735 to 0.745). The agreement was the highest for video clips wherein the wall motion corresponded to the valve motion. However, the agreement fell to unacceptable levels (Kα=0.304; 95% CI 0.287 to 0.321) across video clips wherein the wall motion occurred without the valve motion. There is an overall acceptable interobserver agreement when interpreting cardiac standstill among PEM attendings with an experience of at least 25 previously reported cardiac POCUS scans. However, factors that may influence the lack of agreement include discordances between the wall and valve motion, suboptimal views, and the lack of a formal reference standard. More specific consensus reference standards of pediatric cardiac standstill may help to improve interobserver agreement moving forward and should include more specific details regarding the wall and valve motion.

Intra-observer reproducibility and inter-observer agreement of Fels skeletal age assessments among male tennis players 8–16 years

BackgroundSkeletal age (SA) is an estimate of biological maturity status that is commonly used in sport-related medical examinations. This study considered intra-observer reproducibility and inter-observer agreement of SA assessments among male tennis players.MethodsSA was assessed with the Fels method in 97 male tennis players with chronological ages (CA) spanning 8.7–16.8 years. Radiographs were evaluated by two independent trained observers. Based on the difference between SA and CA, players were classified as late, average or early maturing; if a player was skeletally mature, he was noted as such as an SA is not assigned.ResultsThe magnitude of intra-individual differences between repeated SA assessments were d = 0.008 year (observer A) and d = 0.001 year (observer B); the respective coefficients of variation were 1.11% and 1.75%. Inter-observer mean differences were negligible (t = 1.252, p = 0.210) and the intra-class correlation coefficient was nearly perfect (ICC = 0.995). Concordance of classifications of players by maturity status between observers was 90%.ConclusionFels SA assessments were highly reproducible and showed an acceptable level of inter-observer agreement between trained examiners. Classifications of players by skeletal maturity status based on assessments of the two observers were highly concordant, though not 100%. The results highlight the importance of experienced observers in skeletal maturity assessments.