Limits Of Agreement Research Articles

Accuracy of self-reported height, weight, and waist circumference in poor, rural areas of China

Objectives There are differing opinions and findings about the accuracy of self-reported data. This study sought to assess the accuracy of self-reported height, weight, and waist circumference (WC) data among adult residents in poor, rural areas of China. Methods Multi-stage stratified cluster random sampling was used to sample rural adult residents in Jinzhai County. Intraclass correlations (ICCs), Bland-Altman plots with limits of agreements (LOAs), and Cohen’s kappa were used to examine the accuracy of self-reported data. Multiple linear regression was used to explore the effect of demographic and socioeconomic variables. Results Height was over-reported by an average of 0.67 cm, weight and WC were both under-reported, by an average of 0.004 kg and 1.09 cm, respectively. The ICC for height was 0.92, the ICC for weight was 0.93, and the ICC for WC was 0.93. Bland-Altman plots showed no trend in variability with increasing height, weight, and WC, but 95% of the LOAs were wide, especially for WC. The percentage agreement (kappa) for BMI was 82.10% (0.71), and the percentage agreement for WC was 83.56% (0.66). Overall, obesity, overweight, and increased WC were underestimated by 0.56%, 2.64%, and 6.89%, respectively. Assessments of which demographic and socioeconomic factors were associated with statistically significant differences between self-reported and measured data showed that education (junior school vs. primary or below: β = −0.56, 95% CI [−1.01 to −0.11]) and occupation (others vs. professionals: β = 2.16, 95% CI [0.62–3.70]) were associated with differences between measured and self-reported height; occupation (industrial and commercial service vs. professionals: β = 1.57, 95% CI [0.52–2.61]) was associated with differences between measured and self-reported weight; and gender (women vs. men: β = −1.74, 95% CI [−2.51 to −0.97]) and age (60– vs. 18–44: β = −1.62, 95% CI [−2.88 to −0.36]) were associated with differences between measured and self-reported WC. Conclusions Self-reported height and weight were shown to be reliable in poor, rural areas of China, but self-reported WC should be used with caution. To improve the accuracy of self-reported data, certain subgroups should be targeted for educational interventions.

Analysis of Tumor Proliferation Markers in Early-Stage Luminal Breast Cancer: A Comprehensive Study Using Mitotic Activity Index, Ki-67, and Phosphohistone H3 Expression.

Introduction and Aim: Routinely used proliferation markers such as mitotic activity index (MAI) and Ki-67 index show limited reproducibility due to high interobserver variability in breast cancer assessment. Phosphohistone H3 (PhH3), a novel proliferation marker, is gaining attention in breast cancer research. This study aimed to evaluate the inter-rater agreement among MAI, Ki-67, and PhH3 expressions in early-stage luminal breast cancer and assess the impact of replacing MAI with PhH3 index on tumor histological grading. Materials and Methods: Three pathologists assessed MAI, Ki-67, and PhH3 expressions in 66 early-stage luminal breast cancer specimens. Mitotic Activity Index was scored based on mitotic figures in an area of 2 mm2 while Ki-67 index utilized a 14% threshold for positively stained nuclei. Phosphohistone H3 expression cutoff was set at 13 positive cells per 2 mm2. The inter-rater agreement for the 3 variables was analyzed using Cohen kappa statistics. Results: Among the 3 parameters, the kappa score of the PhH3 expression reflected very strong agreement between the 3 observers (κ = 0.991, 0.907, and 0.916). Only moderate agreement was noted for MAI (κ = 0.898, 0.562, and 0.592) and substantial agreement for Ki-67 index (κ = 0.869, 0.673, and 0.678). Moreover, replacing MAI with PhH3 index led to upgrade of histological grade in 15% to 16% of patients. Conclusion: Our study demonstrated that PhH3 is a more reproducible proliferation marker than MAI and Ki-67. Incorporation of PhH3-based mitotic index in breast cancer grading might reduce the variation in the assessment of histological grade.

Accuracy of O-RADS System in Differentiating Between Benign and Malignant Adnexal Masses Assessed via External Validation by Inexperienced Gynecologists

Objective: To evaluate the accuracy of the O-RADS system in differentiating between benign and malignant adnexal masses, as assessed by inexperienced gynecologists. Methods: Ten gynecologic residents attended a 20 h training course on the O-RADS system conducted by experienced examiners. Following the training, the residents performed ultrasound examinations on patients admitted with adnexal masses under supervision, recording the data in a database that included videos and still images. The senior author later accessed this ultrasound database and presented the cases offline to ten residents for O-RADS rating, with the raters being blinded to the final diagnosis. The efficacy of the O-RADS system by the residents and inter-observer variability were assessed. Results: A total of 201 adnexal masses meeting the inclusion criteria were evaluated, consisting of 136 (67.7%) benign masses and 65 (32.3%) malignant masses. The diagnostic performance of the O-RADS system showed a sensitivity of 90.8% (95% CI: 82.2–96.2%) and a specificity of 86.8% (95% CI: 80.4–91.8%). Inter-observer variability in scoring was analyzed using multi-rater Fleiss Kappa analysis, yielding Kappa indices of 0.642 (95% CI: 0.641–0.643). The false positive rate was primarily due to the misclassification of solid components in classic benign masses as O-RADS-4 or O-RADS-5. Conclusions: The O-RADS system demonstrates high diagnostic performance in distinguishing benign from malignant adnexal masses, even when used by inexperienced examiners. However, the false positive rate remains relatively high, mainly due to the over-interpretation of solid-appearing components in classic benign lesions. Despite this, inter-observer variability among non-expert raters was substantial. Incorporating O-RADS system training into residency programs is beneficial for inexperienced practitioners. This study could be an educational model for gynecologic residency training for other systems of sonographic features.

Deciphering disagreement in the annotation of EU legislation

AbstractThe topic of annotating legal data has received surprisingly little attention. A key challenge of the annotation process is reaching a sufficient agreement between annotators and filtering mistakes from genuine disagreement. This study presents an approach that provides insights into and resolves potential disagreement amongst annotators. It (1) introduces different strategies to calculate agreement levels and compares (2) agreement levels between annotators (inter-annotator agreement) before and after a revision round and (3) agreement levels for annotators who annotate the same texts twice (intra-annotator agreement). The inter-annotator agreement levels are compared to a revision round in which an arbiter corrected the annotator’s labels. The analysis is based on the annotation of EU legislative provisions at two stages (initial annotations, after annotator revisions) and for various tasks (Definitions, References, Quantities, IF-THEN statements, Exceptions, Scope, Hierarchy, Deontic Clauses, Active and Passive Role) by multiple annotators. The results reveal that agreement levels vary based on the stage of measurement (before/after revisions), the nature of the task, the method of assessment, and the annotator combination. The agreement scores - along with some initial measurements—align with those reported in previous research but increase after each revision round. This suggests that annotator revisions can substantially reduce disagreement. Additionally, disagreements were found not only between but also among annotators. This inconsistency does not appear to stem from a lack of understanding of the guidelines or a lack of seriousness in task execution, as evidenced by moderate to substantial inter-annotator agreement scores. These findings suggest that annotators identified multiple valid interpretations, which highlights the complexity of annotating legislative provisions. The results underscore the significance of embracing, addressing, and reporting about (dis)agreement in different ways and at the various stages of an annotation task.

Different Evaluation Strategies of Oncological CT Examinations with Regard to Professional Experience: A Clinical Study Using Eye-tracking.

Contrast-enhanced CT is the standard imaging technique in oncological objectives. Rates of missed pathologies depend on work experience of the respective radiologists. Thus the aim of this study is to analyze the eye movements of professionals while reading CT images in order to evaluate whether the eye-fixation patterns and search strategies of experienced radiologists could explain higher detection rates of pathologies and whether such patterns can be learned.Anonymized images of 10 patients were presented to three medical students and six radiologists with different levels of work experience. During image analysis, ocular fixation positions were recorded using an eye-tracking software tool. The CT scans were analyzed retrospectively, considering the individual course of disease with the issue of successful detection of all pathologies. Visual attention and dwell time of ocular fixation on clinically important abnormalities or areas with pathological findings, general search patterns, and time efficiency were assessed. For statistical analysis, interobserver variability and accuracy of lesion detection were evaluated taking into account individual experience.The results revealed that observer sensitivity depends on work experience due to a more systematic order of inspection and a well-known course of disease, e.g. in case of metastatic spread. The areas of missed pathologies mostly included secondary findings. Inexperienced readers changed the stratification considerably more often and required more time for reporting or detecting pathologies.Our results suggest that experienced radiological physicians reduce their amount of missed findings by looking more systematically at images and by applying a more targeted inspection of clinically important regions. · CT interpretation by radiology residents is faster and less error-prone compared to postgraduate residents. · systematic image analysis is trainable. · engrams tend to be acquired through experience. · Kluge S, Schülke C, Ites HC et al. Different Evaluation Strategies of Oncological CT Examinations with Regard to Professional Experience: A Clinical Study Using Eye-tracking. Fortschr Röntgenstr 2024; DOI 10.1055/a-2452-2180.

Expert level of detection of interictal discharges with a deep neural network.

Deep learning methods have shown potential in automating the detection of interictal epileptiform discharges (IEDs) in electroencephalography (EEG). We compared IED detection using our previously trained deep neural network with a group of experts to assess its potential applicability. First, we performed clinical validation on an internal data set. Seven experts reviewed all EEG studies. Performance agreement between experts and the network was compared at both the EEG and IED levels. All EEG recordings were also processed with Persyst. Subsequently, we performed external validation, with data from four centers, using a hybrid approach, where detections by the deep neural network were reviewed by an expert. In case of disagreement with the original report, the EEG recording was annotated independently by five experts. For internal validation we included 22 EEG studies with IEDs and 28 EEG studies from controls. At the EEG level, our network showed performance similar to that of the experts. For individual IED detection, the sensitivities between experts ranged from 20.7%-86.4%, whereas the sensitivity of our network was 82.5% (confidence interval [CI]: 77.7%-87.4%) at 99% specificity and a false detection rate (FDR) of <.2/min, outperforming Persyst, with 64.6% sensitivity (CI: 61.4%-67.9%) at 98% specificity. External validation in 174 EEG studies demonstrated that all 85 EEG recordings classified as normal in the original report were classified correctly, with an FDR of .10/min. Of the 89 EEG studies with IEDs according to the report, 56 were correctly classified (Cohen's κ = .62). Visual analysis of the remaining 33 EEG recordings showed high interobserver variability among the five experts (Fleiss' κ = .13). Our deep neural network detects IEDs on par with clinical experts. The external validation in a hybrid approach showed substantial agreement with the original report. Disagreement was due mainly to high interobserver variability. Our deep neural network may support visual EEG analysis and assist in diagnostics, particularly when human resources are limited.

Abstract 4144553: Fully Automated Machine Learning Based Echocardiographic Assessment of Global Longitudinal Strain in Breast Cancer Patients Receiving Cardiotoxic Chemotherapy

Background: Global longitudinal strain (GLS) is considered the most sensitive echocardiographic marker for detecting cardiotoxicity and subclinical changes in cardiac function in patients receiving cardiotoxic therapies. Despite literature that has shown less inter-observer variability for GLS compared to left ventricular ejection fraction, there remains significant variability in GLS measurement across observers and vendors. US2.AI is a novel, machine learning-based artificial intelligence (AI) algorithm for automated echocardiographic interpretation. We aim to demonstrate the feasibility of AI-based GLS assessment and hypothesize that AI-based GLS measurement is more reproducible compared to human-based measurement Methods: A total of 27 transthoracic echocardiograms were selected at random from a population of female cardio-oncology breast cancer patients and were run through the US2.AI algorithm twice to calculate GLS. The same echocardiograms were then analyzed by two board-certified cardiologists in a blinded fashion to calculate GLS using QLAB software (Philips). The two methods of GLS measurement (AI versus humans) were compared using linear regression, as assessed by the Pearson correlation coefficient. Reproducibility for each method was assessed using the intra-class correlation coefficient (ICC). Results: A total of 27 studies were analyzed with a mean age of 55.6 ± 11.3 and all patients were female. The feasibility of US2.AI for the assessment of GLS was demonstrated with a strong correlation between AI-based and human reads (r = 0.80, p &lt; 0.00001). The reproducibility of AI-based GLS assessment was superior to that of human readers as assessed by an ICC of 1.0 versus 0.52, respectively. Conclusion: AI-based assessment of GLS was shown to be both feasible and more reproducible compared to expert human readers in a population of breast cancer patients receiving cardiotoxic chemotherapy. Given the important clinical implications of GLS in cardio-oncology patients, more widespread implementation of AI-based strain assessment has the potential to significantly improve the reproducibility of GLS measurement in this unique patient population.

Evaluation of photoplethysmography-based monitoring of pulse rate, interbeat-intervals, and oxygen saturation during high-intensity interval training.

Heart disease patients necessitate precise monitoring to ensure the safety and efficacy of their physical activities when managing conditions such as hypertension or heart failure. This study, therefore, aimed to evaluate the accuracy of photoplethysmography (PPG)-based monitoring of pulse rate (PR), interbeat-intervals (IB-I) and oxygen saturation (SpO2) during high-intensity interval training (HIIT). Between January and March 2024, healthy volunteers were subjected to a cycling HIIT workout with bike resistance increments to evaluate performance within different heart rate ranges. To determine the accuracy of PPG-based measurements for PR, IB-I, and SpO2 using the CardioWatch 287-2 (Corsano Health, the Netherlands), measurements throughout these ranges were compared to paired reference values from the Covidien Nellcor pulse oximeter (PM10N) and Vivalink's wearable ECG patch monitor. Subgroups were defined for Fitzpatrick skin type and gender. In total, 35 healthy individuals participated, resulting in 7183 paired measurements for PR, 22,713 for IB-I, and 41,817 for SpO2. The PR algorithm showed an average root mean square (Arms) of 2.51 beats per minute (bpm), bias at 0.05bpm, and limits of agreement (LoA) from -4.87 to 4.97bpm. The IB-I algorithm achieved an Arms of 23.00ms, a bias of 1.00ms, and LoA from -43.82 to 46.21ms. Finally, the SpO2 algorithm showed an Arms of 1.28%, a bias of 0.13%, and LoA from -2.37% to 2.62%. The results were consistent across different demographic subgroups. This study demonstrates that the PPG-based CardioWatch 287-2 can accurately monitor PR, IB-I, and SpO2 during HIIT. However, further research is recommended to evaluate the algorithm's performance in heart disease patients during demanding exercise.

Agreement of in-ear temperature to core body temperature measures during invasive whole-body cooling for hypothermic circulatory arrest in aortic arch surgery.

Targeted temperature management (TTM) with therapeutic hypothermia (TH) during aortic arch surgery requires valid estimations of core body temperature. The ear canal and epitympanic region might be an easy-to-assess, noninvasive site for the read-out of supra-aortic, cerebral temperature. This observational cohort study comparatively investigated in-ear temperature and different core body temperature (cBT) measurements during TTM/TH for moderate hypothermic circulatory arrest (mHCA) in aortic arch surgery. In total 24 patients (mean age of 56.8 ± 17.5years; six females) were measured using infrared-thermography of the epitympanic region (BTtym), thermistor-based measurements at the esophagus (BTeso; gold standard), at the ear canal (BTear), at the nasopharynx (BTnas), in the bladder (BTves), and in the rectum (BTrec). The data analysis comprised absolute agreement (AA), bias, intraclass correlation coefficient (ICC), and limit of agreement (LoA). The results revealed high AAs of BTtym, BTear, BTnas in reference to BTeso (biases 0.3-0.6°C), with also excellent ICCs > 0.9. BTves and BTrec showed lower AAs, higher biases of + 2.5°C to 3.1°C with moderate ICCs during mHCA. In the phases of rapid temperature changes, the biases and LoAs were higher throughout all BT measurements. Herein, BTtym performed best of all measurement sites. The study informs about the BT dynamics at different body sites during the mHCA procedure. It supports the approach of using minimally invasive in-ear techniques to estimate core body temperature in an intrahospital TTM/TH setting of mHCA.

Factors of interobserver variability in prostate tumor MRI delineation: impact of PI-QUAL score.

Prostate cancer ranks as the second most common cancer in men worldwide. Dose escalation to the tumor and/or the prostate improves biochemical recurrence-free survival. However, interobserver variability in lesion contouring poses a significant limitation to such therapeutic approaches. Therefore, a study of factors influencing this variability is necessary. Three independent readers delineated the index prostate lesion (DIL) using T2w and ADC sequences for each patient. Clinical data were retrospectively collected for all participants. Logistic regression analysis was employed to examine the correlation between clinical features and a mean DICE coefficient > 0.7. Features exhibiting a p value < 0.05 on univariate analysis were subjected to multivariate analysis. The study comprised 68 patients, with a median DICE coefficient of 0.69 (95% CI 0.65-0.71), wherein 42.6% (29/68) attained a mean DICE > 0.7. Univariate analysis identified the PI-QUAL score, maximum diameter of DIL, and mean DIL volume as significant (p < 0.05) predictors. In multivariate analysis, only the PI-QUAL score (p = 0.008) remained statistically associated with a DICE coefficient > 0.7. The PI-QUAL score emerges as the primary predictive factor for minimizing inter-reader variability in intraprostatic dominant lesion segmentation. These findings underscore the importance of considering PI-QUAL scores when devising focal treatment plans. Adoption of a multi-reader approach involving diverse medical specialists (radiologists, radiotherapists, urologists) is advocated, particularly for MRIs with low PI-QUAL scores. Radiotherapy is a major treatment for patients with localized prostate cancer. Dose escalation to the tumor leads to improved cancer control. Precise delineation of the dominant intraprostatic lesion (DIL) remains a limitation to focal treatments. Features influencing inter-reader variability were never evaluated. In this study, we identified that the PI-QUAL score was the sole predictor of the inter-reader delineation variability of the DIL.

Reliable and easy-to-use calculating tool for the Nail Psoriasis Severity Index using deep learning

Since nail psoriasis restricts the patient’s daily activities, therapeutic intervention based on reliable and reproducible evaluation is critical. The Nail Psoriasis Severity Index (NAPSI) is a validated scoring tool, but its usefulness is limited by interobserver variability. This study aimed to develop a reliable and accurate NAPSI scoring tool using deep learning. The tool “NAPSI calculator” includes two parts: nail detection from images and NAPSI scoring. NAPSI was annotated by nine nail experts who are board-certified dermatologists with sufficient experience in a specialized clinic for nail diseases. In the final test set, the “NAPSI calculator” correctly located 137/138 nails and scored NAPSI with higher accuracy than the compared six non-board-certified residents: 83.9% vs 65.7%; P = 0.008 and four board-certified non-nail expert dermatologists: 83.9% vs 73.0%; P = 0.005. The “NAPSI calculator” can be readily used in a clinical situation, contributing to raising the medical practice level for nail psoriasis.

American Foregut Classification is Superior to Hill Classification in Evaluation of the Gastroesophageal Junction

Background: The American Foregut Society (AFS) classification is intended to be more comprehensive and accurate than the Hill grade in assessing the gastroesophageal junction. However, real-world data regarding predictive ability and interobserver variability are limited. Methods: We performed a multi-reader validation study on images collected from patients who underwent endoscopic evaluation of reflux with hernia provocation using both AFS and Hill classification. The primary outcome was inter-reader agreement, assessed using Fleiss K. Secondary outcomes were percentage of patients with pathologic reflux at each Hill grade or AFS grade, correlation between acid exposure time (AET) and individual components of the AFS grade, and difference in predictive ability based on expert versus non-expert graders. Results: Sixty-four eligible patients were identified. Inter-rater reliability was substantial for AFS classification ( K = 0.65) and fair for Hill classification ( K= 0.28). The AFS grade demonstrated moderate overall correlation with AET (ρ = .36) and the Hill grade demonstrated overall weak correlation (ρ = .28). No single component of the AFS score performed significantly better in predicting pathologic reflux than the AFS grade (AFS F = 9.93, P = .0025) and there was no significant difference in overall grading between expert- and non-expert readers (ρ = .38 vs ρ = .29). Conclusion: The AFS score can classify the GEJ with moderate interobserver agreement and is superior to the Hill grade in predicting acid exposure time. AFS grade has better predictive power than the individual components used in scoring and is robust to differences in reader experience.

Comparison of Scheimpflug tomography, Placido disc, and combined Placido Scheimpflug in the measurement of pupil offset in myopic population

IntroductionThis study aimed to compare the consistency of pupil offset measurements obtained using the Pentacam, Keratron Scout, and Sirius devices.MethodsThis retrospective cross-sectional study included 146 young myopic individuals (292 eyes) scheduled for refractive surgery at Dalian Third People’s Hospital between January 2023 and December 2023. Three devices were utilized to measure the chord mu of the pupil deviation along with the Cartesian distances of the X and Y coordinates (Px, Py) associated with the pupil offset. Repeated-measures analysis of variance was used to compare differences in pupil offset acquisition across various devices. Additionally, the intraclass correlation coefficient (ICC) and Bland–Altman plot were utilized to assess the consistency among the three devices.ResultsChord mu, measured using the Pentacam, Keratron Scout, and Sirius devices, were 0.18 ± 0.10, 0.21 ± 0.11, and 0.18 ± 0.11, respectively. The Px values were 0.00 ± 0.14, -0.02 ± 0.16, and -0.01 ± 0.13, respectively, while the Py values were 0.09 ± 0.13, 0.10 ± 0.15, and 0.10 ± 0.13. The ICCs for the three device measurements, chord mu, Px, and Py, were 0.817, 0.900, and 0.855, respectively. When comparing the three devices, the 95% limits of agreement (LoA) for mu and Px measured using the Sirius and Keratron Scout were the narrowest, ranging from −0.15 to 0.08 and −0.11 to 0.13, respectively. Additionally, the 95% LoA for Py measured using the Sirius and Pentacam was the narrowest, ranging from −0.13 to 0.15. The pupil centers in both eyes were predominantly located above the apex of the cornea.ConclusionSirius, Keratron Scout, and Pentacam have good consistency in pupil shift measurement in young myopic patients, and the three devices can be used as references in clinical practice.

Analysis of the sensitivity of high-grade squamous intraepithelial lesion Pap diagnosis and interobserver variability with the Hologic Genius Digital Diagnostics System.

Artificial intelligence (AI)-based systems are transforming cytopathology practice. The aim of this study was to evaluate the sensitivity of high-grade squamous intraepithelial lesion (HSIL) Papanicolaou (Pap) diagnosis assisted by the Hologic Genius Digital Diagnostics System (GDDS). A validation study was performed with 890 ThinPrep Pap tests with the GDDS independently. From this set, a subset of 183 cases originally interpreted as HSIL confirmed histologically were included in this study. The sensitivity for detecting HSIL by three cytopathologists was calculated. Most HSIL cases were classified as atypical glandular cell/atypical squamous cell-high grade not excluded (AGC/ASC-H) and above by all cytopathologists. Of these cases, 11.5% were classified as low-grade squamous intraepithelial lesion (LSIL) by pathologist A (P-A), 6% by pathologist B (P-B), and 5.5% by pathologist C (P-C); 3.8%, 2.7%, and 1.6% of these cases were classified as atypical squamous cell of unknown significance (ASC-US) by P-A, P-B, and P-C, respectively. The sensitivity for detection of cervical intraepithelial neoplasia 2 and above (CIN2+) lesions was 100% if ASC-US and above (ASC-US+) abnormalities were counted among all three pathologists. The sensitivity for detection of CIN2+ lesions was 84.7%, 91.3%, and 92.9% by P-A, P-B, and P-C, respectively, for ASC-H and above abnormalities. The Kendall W coefficient was 0.722, which indicated strong agreement between all pathologists. New-generation AI-assisted Pap test screening systems such as the GDDS have the potential to transform cytology practice. In this study, the GDDS aided in interpreting HSIL in ThinPrep Pap tests, with good sensitivity and agreement between the pathologists who interacted with this system.

Repeatability and agreement of multispectral refraction topography in school children before and after cycloplegia

BackgroundThe purpose of this study was to evaluate the repeatability and agreement of multispectral refraction topography (MRT) in measuring retinal refraction before and after cycloplegia in children. The results of this study will provide valuable insights into the accuracy and reliability of MRT as a tool for assessing retinal refraction in pediatric patients.MethodsChildren aged 7 to 18 years old were recruited for this prospective research. The central and peripheral retinal refraction was measured three times using multispectral refraction topography (MRT) before and after cycloplegia. The retinal deviation value (RDV) was used to describe the average peripheral refractive error of the retina. In addition, objective refraction (OR) and subjective refraction (SR) measurements were also performed.ResultsA total of 60 children with a mean age of 10.50 ± 1.81 years were enrolled. Before cycloplegia, all the central and peripheral retinal refraction parameters showed good repeatability with the lowest intraclass correlation coefficient (ICC) being 0.78 in the retinal deviation value from 45° eccentricity to 53° of the retina (RDV 45–53). After cycloplegia, the repeatability of MRT was significantly enhanced (lowest ICC = 0.91 in RDV-I). The 95% limits of agreement (LoA) of the central refraction and OR ranged from − 2.1 to 1.8 D before cycloplegia, and from − 1.69 to 0.27 D after cycloplegia. The 95% LoA of the central refraction and SR ranged from − 1.57 to 0.36 D after cycloplegia. All the 95% LoA demonstrated high agreement.ConclusionsThe MRT shows high agreement with autorefractometry and experienced optometrist in measuring central refraction. Additionally, the MRT provides good repeatable measurements of retinal peripheral refraction before and after cycloplegia in schoolchildren.

Real-Time Accuracy Evaluation of Arterial Catheter Transducer Systems

IntroductionArterial pressure is currently monitored in ICU with a catheter–transducer fluid line. This fluid filled tubing line distorts the original waveform due to its dynamic characteristics (natural frequency, Fn, and damping coefficient, z), introducing potentially significant errors when calculating the cardiac output from pulse contour signal analysis. MethodsIn our study, we cross-compared Fn and z obtained with our new Fast External Pressure Test (FEPT) and with the Fast Flush Test (FFLT), to the reference technique (Sine wave variable Frequency Analysis Test - SFAT). It was carried on a testbench for 48 hours. Fn and z were measured using the three techniques with two fluid-filled tubing lines (standard, STD, and blood conserving device, BCD). ResultsFn measurements with FEPT and FFLT present similar biases (0.79 vs 0.83 Hz), but lower variability for FEPT, with limits of agreement (LOA) ranging from −3.35 to +4.99 Hz for FFLT vs −2.29 to +3.86 Hz (p<0.0001) for FEPT. For the measurement of z, FEPT has a bias of 0.047 and LOA of −0.063 to +0.157, much lower (p<0.0001) than those measured with the FFLT (bias 0.139 and LOA −0.028 to +0.306). ConclusionWhen automated, the FEPT method will detect potential situations of over/under estimations occurrences. This will prevent false alarms, alarm fatigue and therefore consequences on patient care. Eventually, FEPT turns to be more accurate than FFLT, less scattered, less time-consuming, less invasive and so well suited for use in clinical settings.

Agreement and Reliability of Transpalpebral Tonometers with Goldmann Applanation Tonometer – A Systematic Review and Meta-Analysis

TopicThe systematic review and meta-analysis consolidates the extant body of evidence comparing reliability and agreement between Transpalpebral Tonometers (TT) and Goldmann Applanation Tonometer (GAT). Clinical RelevanceWith a global prevalence of 3.54 percent, glaucoma stands as the second leading cause of preventable blindness. Projections indicate a rise to 111 million cases by 2040. Existing literature presents inconsistent findings while comparing TT and GAT. The derivation of summary estimates assessing their agreement holds significance given TT’s multifaceted applicability in clinical, community, and home settings. MethodsSystematic review was conducted using PubMed, Cochrane Library, and Google Scholar from January 2000 to December 2022. Two reviewers independently evaluated, enumerated, and extracted studies and data based on eligibility criteria. The Quality Assessment for Diagnostic Accuracy Studies (QUADAS-2) checklist was used to assess study quality. The summary measures were pooled using the random effects model as mean difference (MD), and 95% LoA (Limits of Agreement). We assessed heterogeneity using the I2 statistic. The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO - CRD42022321693). ResultsA total of 26 methods comparison studies (3577 eyes) were included in the meta-analysis. The overall random-effects MD (TT – GAT), SD and 95% LoA for IOP were -0.70 ± 4.15 mmHg (-8.83 to 7.43 mmHg). In the subgroup analysis based on index test devices used, Easyton showed the lowest MD, SD (-0.29 ± 2.35 mmHg), and 95% LoA (-4.90 to 4.32 mmHg). In the univariate meta-regression model, we found that, on average, studies examining normal eyes reported a statistically significant lower mean difference of 2.67 mmHg (95% CI: 0.27 to 5.07 mmHg; P = 0.03) between TT and GAT, compared to studies that assessed eyes with mixed ocular condition. ConclusionIn the current meta-analysis, we found a small MD in the measured IOP between the two tonometers. However, given the high heterogeneity and a wider LoA, it is not advisable to use TT interchangeably with GAT.

Predictability of Keratorefractive Lenticule Extraction Is Equal to Variance of Preoperative Manifest Refraction Measurement.

To compare variance of manifest refraction (MR) measurement and variance of refractive outcomes after keratorefractive lenticule extraction (KLEx) in the same cohort. This was a retrospective study of consecutive patients attending at least three preoperative refractions before undergoing KLEx. All manifest refractions were performed according to a standard protocol. Additionally, automated refraction, spectacles prescription, wavefront refraction, and tomography/topography were also considered when determining the treatment refraction. Variance of postoperative outcomes was compared to variance of MR measurement. The difference in MR of 153 eyes between the first and last preoperative visit showed a mean pairwise absolute difference of 0.22 ± 0.22 diopters (D) (range: 0.00 to 1.25 D) in spherical equivalent (SE). The 95% limit of agreement (LoA) was within 0.73 D for sphere, 0.60 D for cylinder, and 0.61 D for SE. Standard deviation (SD) for corrected distance visual acuity (CDVA) was 0.06 logarithm of the minimum angle of resolution (logMAR) and the 95% LoA was within 1.3 Snellen lines. The differences between reproducibility of the two last preoperative MR and the variance of postoperative outcomes after KLEx (ie, the isolated repeatability of corneal lenticule extraction) was SD of 0.13 D for sphere, cylinder, and SE, and 11 degrees in axis, respectively. Predictability of KLEx was equal to variance of refraction measurement. Isolated repeatability of the laser treatment was better than the reproducibility of MR. Thus, the authors have shown for the first time that precision of KLEx has reached such a level today that further improvement is limited by MR measurements. [J Refract Surg. 2024;40(11):e814-e823.].

Measuring up: Ensuring Intra- and Interobserver Reliability in Stretched Penile Length with the SPLINT Technique

ABSTRACT Background: A discrepancy between the true and measured value of stretched penile length (SPL) may be a result of errors that can either be systematic or random. Hence, it becomes important to focus on the quality of measurements to prevent any iatrogenic harm to the patients. Objective: The objective of this study was to assess the magnitude of intra- and interobserver variations in the measurement of SPL with the SPLINT technique. Materials and Methods: SPL was measured prospectively in a cohort of 449 boys aged 0–14 years including 68 infants (substratified into Group I: &gt;4 years, Group II: 4–8 years, and Group III: &gt;8 years) with the SPLINT technique by expert (E: E1 and E2) and trainee (T: T1 and T2) surgeons after completing a three-tiered training module. Intra- and interobserver variability was assessed through descriptive statistics, intraclass correlation (ICC), relative technical error of measurement (rTEM), and reliability or R (%). Results: Intraobserver variability: the mean difference between the two readings (E1 and E2) is 0.08 cm (95% confidence interval [CI]: 0.073–0.087), ICC was 0.998 (95% CI: 0.997–0.998), and intraobserver variability ≤0.1 cm in 85% of the participants (n = 370 of 433). The rTEM and reliability (%) were 1.82% and 98.1% (Group I), 1.65% and 98.9% (Group II), and 1.09% and 99.7% (Group III), respectively. The intraobserver variability was observed to be inversely proportional to the age of the participants (correlation coefficient = −0.56). Interobserver variability was calculated separately for expert versus trainee and trainee versus trainee (T-vs-T) measurements. For expert versus expert, ICC, rTEM, and reliability (%) were 0.984, 2.4%, and 96.8% (Group 1), 0.992, 2.07%, and 98.3% (Group 2), and 0.997, 1.38%, and 99.05% (Group 3), respectively. A similar pattern of variability was observed for T-vs-T measurements. The reliability (%) of the SPL by experts is consistently more than that of trainees across all age groups; however, the difference ameliorates with the age of participant. Conclusions: The study has validated the SPLINT technique by demonstrating a high level of intra- and interobserver reliability. The adequacy of the training modules for SPL measurements described in this study has also been established. Evidence that the SPL can be used as an objective marker of penile dimensions is herewith furnished.

Radiologists vs. AI-based software: Predicting Lymph Node Metastasis and Prognosis in Lung Adenocarcinoma from CT under Various Image Display Conditions

PurposeTo compare the variability of quantitative values from lung adenocarcinoma CT images independently assessed by two radiologists and AI-based software under different display conditions, and to identify predictors of pathological lymph node metastasis (LNM), disease-free survival (DFS), and overall survival (OS). MethodsPreoperative CT images of 307 patients were displayed under four conditions: lung-1, lung-2, mediastinum-1, and mediastinum-2. Two radiologists (R1, R2) measured total diameter (tD) and the longest solid diameter (sD) under each condition. The AI-based software automatically detected lung nodules, providing tD, sD, total volume (tV), and solid volume (sV). ResultsAll measurements by R1 and R2 with AI-based software were identical. Four out of the eight measurements showed significant variation between R1 and R2. For LNM, multivariate logistic regression identified significant indicators including sD at mediastinum-2 of R1, sD at mediastinum-1 and mediastinum-2 of R2, tV, and the proportion of sV to tV (sV/tV) of AI-based software. For DFS, multivariate Cox regression identified sD at lung-1 of R1, the proportions of sD to tD at lung-2 of R1, sD at lung-2 and mediastinum-1 of R2, tV, and sV/tV of AI-based software as significant. For OS, multivariate Cox regression identified sD at lung-1 and mediastinum-2 of R1, tD at lung-2 of R2, sD at mediastinum-1 of R2, sV, and sV/tV of AI-based software as significant. ConclusionRadiologists’ CT measurements were significant predictors of LNM and prognosis, but variability existed among radiologists and display conditions. AI-based software can provide accurate and reproducible indicators for predicting LNM and prognosis.Micro-abstract: Accurate measurement of lung adenocarcinoma is crucial for determining treatment plan and predicting prognosis. However, inter-observer variability and display conditions can affect these measurements. We compared tumor size measurements between radiologists and commercially available AI-based software using preoperative CT images from 307 cases and evaluated their predictive value for lymph node metastasis, disease-free survival, and overall survival. We concluded that while radiologist measurements showed inter-observer variability, AI-based software provided accurate and reproducible prognostic indicators.