Moderate Inter-rater Agreement Research Articles

Interrater reliability of the modified prone instability test for lumbar segmental instability in individuals with mechanical low back pain

ABSTRACT Objective The purpose of this study was to establish the interrater reliability of measures obtained with a novel Modified Prone Instability Test (mPIT), which, like the original Prone Instability Test (PIT), is proposed to identify lumbar segmental instability. The mPIT has clinical feasibility advantages to the PIT, but its psychometric properties are yet to be determined. Design Repeated measures (test-retest) design, methods study Methods The mPIT was administered by two blinded testers, an orthopedic physical therapy resident with < 1 year experience and board-certified orthopedic specialist physical therapist with >25 years’ experience. Procedures were administered at an outpatient physical therapy clinic of a tertiary Medical Center. Participants included 50 adults (≥18 years old) with mechanical low back pain and no radicular (below the knee) symptoms (mean age 50.7 years, 66% female, 76% reported previous episodes of low back pain). Interrater reliability was measured via Fleiss’ kappa coefficient. Results Assessments of the mPIT had moderate interrater agreement (κ = .579 [95% CI = .302 to .856], p < .001.) Conclusion Measures obtained using the mPIT demonstrated moderate interrater reliability between a new graduate and an experienced clinician, which aligns with several studies examining interrater reliability of the original PIT. Further study examining comparative validation of the mPIT with other lumbar instability measures is warranted.

Influence of intercostal muscles contraction on sonographic evaluation of lung sliding: a physiological study on healthy subjects

ObjectivesTo investigate the following: (a) effects of intercostal muscle contraction on sonographic assessment of lung sliding and (b) inter-rater and intra-observer agreement on sonographic detection of lung sliding and lung pulse.MethodsWe used Valsalva and Muller maneuvers as experimental models in which closed glottis and clipped nose prevent air from entering the lungs, despite sustained intercostal muscles contraction. Twenty-one healthy volunteers underwent bilateral lung ultrasound during tidal breathing, apnea, hyperventilation, and Muller and Valsalva maneuvers. The same expert recorded 420 B-mode clips and 420 M-mode images, independently evaluated for the presence or absence of lung sliding and lung pulse by three raters unaware of the respiratory activity corresponding to each imaging.ResultsDuring Muller and Valsalva maneuvers, lung sliding was certainly recognized in up to 73.0% and up to 68.7% of imaging, respectively, with a slight to fair inter-rater agreement for Muller maneuver and slight to moderate for Valsalva. Lung sliding was unrecognized in up to 42.0% of tidal breathing imaging, and up to 12.5% of hyperventilation imaging, with a slight to fair inter-rater agreement for both. During apnea, interpretation errors for sliding were irrelevant and inter-rater agreement moderate to perfect. Even if intra-observer agreement varied among raters and throughout respiratory patterns, we found it to be higher than inter-rater reliability.ConclusionsIntercostal muscles contraction produces sonographic artifacts that may simulate lung sliding. Clinical studies are needed to confirm this hypothesis. We found slight to moderate inter-rater agreement and globally moderate to almost perfect intra-observer agreement for lung sliding and lung pulse.Trial registrationClinicalTrials.gov registration number.NCT 02386696.

Evaluator's alignment as an important indicator of adequacy of the criteria and assessment procedure for recognizing the good practice in public health.

Public health interventions aim to reduce the burden of chronic non-communicable diseases. Implementing evidence-based interventions that are proven to be successful and effective is widely recognized as the best approach to addressing public health challenges. To avoid the development and implementation of less effective or successful or even harmful practices, clear criteria for the assessment of practices, that consider different dimensions of the interventions in public health, are needed. The main aim of the research was to test our Criteria and assessment procedure for recognizing good practices in the field of public health by estimating the consistency between the evaluators and thereby gaining insight into the adequacy and reliability of the criteria as well as to check how the evaluators understand the criteria and methodology and if it is properly used in assessing the interventions. The assessment of the interventions took place from 2021 to 2022. The individual evaluator's scores on the scale from 1 to 5 for each specific sub-criterion were collected, which was followed by a panel discussion to reach a final score for each sub-criterion. The inter-rater agreement was measured using percent overall agreement and Fleiss' kappa coefficient. We found moderate inter-rater agreement on the level of the assessment criteria group. The lowest agreement was observed for the effectiveness and efficiency sub-criteria group, which also received the lowest scores from the evaluators. Challenges identified with the scoring process were due to the descriptive 1 to 5 scale and the varying specificity of the criteria. The results showed that studying consistency between evaluators can highlight areas for improvement or adjustment in the assessment criteria and enhance the quality of the assessment instrument. Therefore, such analysis would be useful part of both newly and well-established health promotion and prevention program registries.

Objective movement asymmetry in horses is comparable between markerless technology and sensor-based systems.

A markerless artificial intelligence (AI) system for lameness detection has recently become available but has not been extensively compared with commonly used inertial measurement unit (IMU) systems for detecting asymmetry under field conditions. Comparison of classification of asymmetric limbs under field conditions and comparison of normalised asymmetry data using a markerless AI system (SleipAI; recorded on a tripod mounted iPhone 14pro [SL]); the Equinosis Q Lameness Locator (LL); the EquiMoves (EM); and subjective evaluation (SE). Descriptive clinical study. Straight line trot data were collected from 52 client-owned horses in regular training. Limbs were categorised as symmetric or asymmetric. Number of analysed strides were compared with Wilcoxon's each pairs test. Inter-rater reliability in classification of asymmetric limbs was assessed with Light's Kappa. Bland Altman analysis of normalised asymmetry data was performed. Data from 41 horses were included. Most horses showed mild asymmetry. The EM analysed significantly more strides than the other systems, both for forelimbs and for hindlimbs (53 ± 11 strides for both, respectively; p < 0.006). The LL analysed significantly more hindlimbs strides (45 ± 13) than the SL (27 ± 6; p < 0.001). Moderate inter-rater agreement for asymmetry classification was found between systems (k = 0.59 forelimbs; 0.44 hindlimbs); agreement decreased when including the SE. For the normalised asymmetry data, the strongest agreement was found between the two IMU systems. Horses were assessed during straight-line trot only. The objective systems were comparable in classification of asymmetric limbs under field conditions when using defined asymmetry thresholds. Discrepancies stemmed largely from the imposed thresholds (i.e., systems largely identified same-side asymmetry). Overall, the strongest agreement was found between LL and EM. The SL analysed significantly fewer hindlimb strides than the LL and EM which could represent a limitation of the Sleip AI.

Evaluating the Role of Ultrasound in Prostate Cancer trial – phase 1: Early experience of micro-ultrasound in the United Kingdom

Purpose: The purpose of this study was to evaluate if the use of micro-ultrasound can detect clinically significant prostate pathology when compared to histology obtained during a transperineal prostate biopsy. Methods: Patients suspected of having prostate cancer, who had a pre-biopsy magnetic resonance imaging and could tolerate a transrectal examination, were prospectively recruited. All patients had a micro-ultrasound scan prior to their biopsy. The findings of magnetic resonance imaging, micro-ultrasound and histology were risk stratified in accordance with local pathways. Comparison of assigned risk scores was made using histology as the reference standard. Results: Data from 101 patients were evaluated. Histology showed that clinically significant prostate cancer was detected in 48.5% ( n = 49/101) of patients. Moderate inter-rater agreement was found in both magnetic resonance imaging and micro-ultrasound with К of 0.31 in both modalities. High-risk findings were identified in 81% ( n = 82/101) patients at magnetic resonance imaging and in 66% ( n = 67/101) patients at micro-ultrasound. Sensitivity and specificity of magnetic resonance imaging were found to be 87% and 34.6% and for micro-ultrasound 73.3% and 53.8%, respectively. Conclusion: A limitation of this study was that the biopsy was not performed with micro-ultrasound which may have resulted in unidentified cancers and lowered the apparent accuracy of the technique. However, we conclude that while micro-ultrasound was diagnostic, magnetic resonance imaging demonstrated higher sensitivity in our local population and remains the pre-biopsy imaging modality of choice. However, the higher specificity of micro-ultrasound identified does indicate that it may be of value when magnetic resonance imaging is contraindicated. The role of micro-ultrasound, within an active surveillance pathway for prostate cancer, warrants further investigation.

An ASER AI/ML expert panel formative user research study for an interpretable interactive splenic AAST grading graphical user interface prototype.

The AAST Organ Injury Scale is widely adopted for splenic injury severity but suffers from only moderate inter-rater agreement. This work assesses SpleenPro, a prototype interactive explainable artificial intelligence/machine learning (AI/ML) diagnostic aid to support AAST grading, for effects on radiologist dwell time, agreement, clinical utility, and user acceptance. Two trauma radiology ad hoc expert panelists independently performed timed AAST grading on 76 admission CT studies with blunt splenic injury, first without AI/ML assistance, and after a 2-month washout period and randomization, with AI/ML assistance. To evaluate user acceptance, three versions of the SpleenPro user interface with increasing explainability were presented to four independent expert panelists with four example cases each. A structured interview consisting of Likert scales and free responses was conducted, with specific questions regarding dimensions of diagnostic utility (DU); mental support (MS); effort, workload, and frustration (EWF); trust and reliability (TR); and likelihood of future use (LFU). SpleenPro significantly decreased interpretation times for both raters. Weighted Cohen's kappa increased from 0.53 to 0.70 with AI/ML assistance. During user acceptance interviews, increasing explainability was associated with improvement in Likert scores for MS, EWF, TR, and LFU. Expert panelists indicated the need for a combined early notification and grading functionality, PACS integration, and report autopopulation to improve DU. SpleenPro was useful for improving objectivity of AAST grading and increasing mental support. Formative user research identified generalizable concepts including the need for a combined detection and grading pipeline and integration with the clinical workflow.

In-person validation of the Ichthyosis Scoring System.

Ichthyoses are a heterogeneous group of skin disorders characterized by scaling and erythema. Recognizing the variability of scale and erythema by region and ichthyosis subtype, we developed the Ichthyosis Scoring System (ISS) to quantify severity. We previously found ISS to have high inter- and intrarater reliability in evaluating photographic images. To confirm ISS clinical utility, we examined its performance at the 2022 Foundation for Ichthyosis and Related Skin Types conference. Sixty-five participants were evaluated by 3 of 9 medical professionals trained to score ichthyosis scale and erythema using ISS. Intrarater and interrater intraclass correlation coefficients (ICC) were analyzed using one-way and two-way random effects models, respectively. Intrarater reliability was excellent (ICC = 0.931, 95% CI, 0.921-0.940) for scale and good (ICC = 0.876, 95% CI, 0.853-0.899) for erythema scoring. Compared to photo validation with excellent intrarater reliability ratings for both scale (ICC = 0.956, 95% CI, 0.925-0.974) and erythema (ICC = 0.913, 95% CI, 0.855-0.949), ISS demonstrated equivalent reliability for live use. Overall interrater reliability for 10 body sites showed excellent (ICC >0.9) and good (ICC >0.75) agreement and consistency for both scale and erythema. Palms were an exception, demonstrating moderate (ICC >0.5) interrater agreement and consistency for erythema evaluation. ISS is a reliable measure of global and regional ichthyosis severity during in-person evaluations. Ease-of-use, accessibility, and content validity in both live and photographic evaluation endorse ISS as a standard for ichthyosis severity analysis.

Variability in clinical assessment of clade IIb mpox lesions

ObjectivesThe ongoing global outbreak of mpox, caused by clade IIb mpox virus, poses significant challenges in accurately categorizing and assessing the diversity of lesions. With lesion resolution being a key endpoint in clinical trials and observational studies, it is essential to evaluate the inter-rater reliability and agreement of clade IIb mpox lesion assessment among clinicians. MethodsClinicians experienced in clade IIb mpox disease were surveyed online with 20 lesion images. They categorized lesions into active, crusted, resolved, or unclassifiable groups. Reliability was assessed with Fleiss' kappa and agreement with proportion of exact agreement. ResultsFifty-three clinicians completed the survey, with a median self-reported confidence rating of 7 (on a scale of 1 to 10) in assessing mpox lesions. The inter-rater reliability was found to be moderate, with a Fleiss' kappa coefficient of 0.417 (P <0.05, 95% confidence interval: 0.409-0.425). The inter-rater agreement was 61%. ConclusionThis study demonstrates moderate inter-rater reliability and agreement in clade IIb mpox lesion assessment among clinicians. The findings emphasize the importance of standardizing lesion classification systems to facilitate clinical care (e.g., decision to start treatment) and public health (e.g. isolation) decisions and a need to explore alternative endpoints for clinical trials.

Psychometric Properties and Clinical Usefulness of the Spanish Version of the Toe Walking Tool.

The Toe Walking Tool (TWT) was developed in Australia as a valid and reliable screening tool for children who toe-walk. However, psychometric properties of the Spanish version of the TWT have not been studied. The aim of this study was to assess psychometric properties and clinical usefulness of the Spanish version of the TWT. A cross-sectional study was conducted. Twelve children were assessed with the TWT. Intrarater and interrater reliability and agreement were calculated using the intraclass correlation coefficient (ICC) and the Fleiss kappa method for multiple raters. Internal consistency and construct validity were assessed with the Kuder-Richardson formula 20 coefficient and known-group methods, respectively. Sensitivity and specificity were analyzed using the receiver operating characteristic curve. The Content Validity Index was calculated to determine clinical usefulness. An excellent intrarater (ICC = 1) and interrater reliability (ICC = 0.8), moderate interrater agreement (Fleiss kappa, 0.6), strong internal consistency (Kuder-Richardson formula 20 coefficient, 0.86), and very good construct validity were found. Sensitivity and specificity results were appropriate (area under the curve, 0.845). The Content Validity Index was 0.906, suggesting high usefulness. The Spanish version of the TWT was found to be a valid, reliable and useful screening tool for children who toe-walk, providing evidence for its recommendation.

Evaluating the impact of a digitally implemented subjective standard on professional rugby union player management decision-making

Using a pre-post-test design, this study evaluated the impact of implementing a standard on the reliability of player management decision-making within a professional rugby union environment. Five practitioners from a High-Performance Unit (HPU) rated 22 instances of Global Positioning System (GPS)–based external training load information of 14 players across the 2021–2022 season. This rating was whether a peak/trough/normal exposure in load had occurred. The ratings were repeated at four time points (separated by 2 weeks) before (Pre1, Pre2) and after (Post1, Post2) implementing a consensus statement as a subjective standard (using a dashboard) developed previously within the same environment to identify peaks/troughs in player external training loads. Inter-rater agreement between individuals at each voting round was assessed using Light's Kappa, while pre-post-standard intra-rater agreement was determined from Cohen's Kappa (both with 95% confidence intervals). Changes to dashboard usability from implementing the standard were assessed by administering the System Usability Scale to 11 HPU staff at the four time points. Pre-standard moderate inter-rater agreement (Pre1: 0.53 (0.36–0.69), Pre2: 0.60 (0.42–0.77)) increased to almost perfect agreement (Post1: 0.74 (0.57–0.89), Post2: 0.90 (0.79–1)) post-standard. The intra-rater agreement of 2/5 participants was almost perfect post-standard, while it remained within substantial levels for the others. A linear mixed model ( χ2(3) = 8.85, p = 0.03) illustrated a slight increase in dashboard usability after incorporating the standard (Pre1: 84.09, Pre2: 81.36; Post1: 87.73, Post2: 87.27). Overall, the results highlighted that the subjective standard enhanced reliability of practitioner agreement for the selected decision.

Accuracy and Reliability of Left Atrial Appendage Morphology Assessment by Three-Dimensional Transesophageal Echocardiographic Glass Rendering Modality: A Comparative Study With Computed Tomography

Contrast-enhanced computed tomography is the reference-standard imaging technique to assess left atrial appendage (LAA) morphology. The aim of this study was to evaluate the accuracy and reliability of two-dimensional and new three-dimensional (3D) transesophageal echocardiographic rendering modalities in assessing LAA morphology. Seventy consecutive patients who underwent both computed tomography and transesophageal echocardiography (TEE) were retrospectively enrolled. The traditional LAA morphology classification system (LAAcs; chicken wing, cauliflower, cactus, and windsock) and a new simplified LAAcs based on the LAA bend angle were used for the analysis. LAA morphology was independently assessed by two trained readers using three different modalities: two-dimensional TEE, 3D TEE with multiplanar reconstruction, and a new 3D transesophageal echocardiographic rendering modality with improved transparency (Glass). The new LAAcs and traditional LAAcs were compared in terms of intra- and interrater reliability. With the new LAAcs, two-dimensional TEE was fairly accurate in identifying LAA morphology (κ=0.43, P<.05), with moderate interrater (κ=0.50, P<.05) and substantial intrarater (κ=0.65, P<.005) agreement. Three-dimensional TEE showed higher accuracy and reliability: 3D TEE with multiplanar reconstruction had almost perfect accuracy (κ=0.85, P<.001) and substantial (κ=0.79, P<.001) interrater reliability, while 3D TEE with Glass had substantial accuracy (κ=0.70, P<.001) and almost perfect (κ=0.84, P<.001) interrater reliability. Intrarater agreement was almost perfect for both 3D transesophageal echocardiographic modalities (κ=0.85, P<.001). Accuracy was considerably lower when the traditional LAAcs was used, with 3D TEE with Glass being the most reliable technique (κ=0.75, P<.05). The new LAAcs showed higher inter- and intrarater reliability compared with the traditional LAAcs (interrater, κ=0.85 vs κ=0.49; intrarater, κ=0.94 vs κ=0.68; P<.05). Three-dimensional TEE is an accurate, reliable, and feasible alternative to computed tomography in assessing LAA morphology with the new LAAcs. The new LAAcs shows higher reliability rates than the traditional one.

Inter-rater concordance of basal cell carcinoma subtypes: influences on reporting format and opportunities for further classification modifications

Diagnosis of basal cell carcinoma (BCC) higher risk subtypes influences management strategies because of their propensity to recur locally. Subtyping is prone to inter-observer variability, and subtyping definitions are inconsistently applied. This study sought to compare the interobserver reproducibility of individual BCC subtypes using the 4th edition World Health Organization (WHO) Classification of Skin Tumours (CoST) definitions, with classification into lower and higher risk histological subtype groups. Ninety-one BCC cases were rated by seven pathologists, noting the presence of BCC subtype(s), and providing a higher or lower risk subtype grouping per case. Raters were provided with definitions as per the 4th edition WHO CoST for 10 listed BCC subtypes. Surgical specimen type was noted. Subgroup analysis was performed to exclude cases when the tumour deep front was not well visualised, or there was tangential sectioning (n= 6). Light's kappa was used to assess inter-rater reliability. From the total group (n= 91), five BCC subtypes showed a sufficient number of ratings for computing a κ statistic. From these five subtypes, superficial subtype showed substantial inter-rater agreement (κ= 0.64), and the other four subtypes showed moderate inter-rater agreement [nodular (κ= 0.45), sclerosing/morphoeic (κ= 0.45), infiltrating (κ= 0.49) and micronodular (κ= 0.57)]. Two-tiered rating into either higher or lower risk subtype showed substantial inter-rater agreement (κ= 0.72). Our results suggest a need to more precisely define BCC subtypes. We suggest reporting BCC subtype using a two-tiered risk grouping, followed by specific subtypes present. Further studies examining the inter-rater reliability of less common BCC subtypes are required.

Colorectal dysplasia in chronic inflammatory bowel disease: a contemporary consensus classification and interobserver study

The clinical management of patients with dysplasia in chronic inflammatory bowel disease (IBD) is currently guided by Riddell et al.'s grading system (negative, indefinite, low grade, high grade) from 1983 which was based primarily on nuclear cytoarchitectural characteristics. Although most dysplasia in IBD resembles sporadic adenomas morphologically, other distinctive potential cancer precursors in IBD have been described over time. Recognizing the need for a updated comprehensive classification for IBD-associated dysplasia, an international working group of pathologists with extensive clinical and research experience in IBD devised a new classification system and assessed its reproducibility by having each participant assess test cases selected randomly from a repository of electronic images of potential cancer precursor lesions. The new classification system now encompasses three broad categories and nine sub-categories: 1) intestinal dysplasia (tubular/villous adenoma-like, goblet cell deficient, crypt cell, traditional serrated adenoma-like, sessile serrated lesion-like and serrated NOS), 2) gastric dysplasia (tubular/villous and serrated), and 3) mixed intestinal-gastric dysplasia. In the interobserver analysis, 67% of the diagnoses were considered definitive and achieved substantial inter-rater agreement. The key distinctions between intestinal and gastric lesions and between serrated and non-serrated lesions achieved substantial and moderate inter-rater agreement overall, respectively, however, the distinctions among certain serrated sub-categories achieved only fair agreement. Based on the Riddell grading system, definite dysplasia accounted for 86% of the collective responses (75% low grade, 11% high grade). Based on these results, this new classification of dysplasia in IBD can provide a sound foundation for future clinical and basic IBD research.

Development and validation of a machine learning ASA-score to identify candidates for comprehensive preoperative screening and risk stratification

ObjectiveThe ASA physical status (ASA-PS) is determined by an anesthesia provider or surgeon to communicate co-morbidities relevant to perioperative risk. Assigning an ASA-PS is a clinical decision and there is substantial provider-dependent variability. We developed and externally validated a machine learning-derived algorithm to determine ASA-PS (ML-PS) based on data available in the medical record. DesignRetrospective multicenter hospital registry study. SettingUniversity-affiliated hospital networks. PatientsPatients who received anesthesia at Beth Israel Deaconess Medical Center (Boston, MA, training [n = 361,602] and internal validation cohorts [n = 90,400]) and Montefiore Medical Center (Bronx, NY, external validation cohort [n = 254,412]). MeasurementsThe ML-PS was created using a supervised random forest model with 35 preoperatively available variables. Its predictive ability for 30-day mortality, postoperative ICU admission, and adverse discharge were determined by logistic regression. Main resultsThe anesthesiologist ASA-PS and ML-PS were in agreement in 57.2% of the cases (moderate inter-rater agreement). Compared with anesthesiologist rating, ML-PS assigned more patients into extreme ASA-PS (I and IV), (p < 0.01), and less patients in ASA II and III (p < 0.01). ML-PS and anesthesiologist ASA-PS had excellent predictive values for 30-day mortality, and good predictive values for postoperative ICU admission and adverse discharge. Among the 3594 patients who died within 30 days after surgery, net reclassification improvement analysis revealed that using the ML-PS, 1281 (35.6%) patients were reclassified into the higher clinical risk category compared with anesthesiologist rating. However, in a subgroup of multiple co-morbidity patients, anesthesiologist ASA-PS had a better predictive accuracy than ML-PS. ConclusionsWe created and validated a machine learning physical status based on preoperatively available data. The ability to identify patients at high risk early in the preoperative process independent of the provider's decision is a part of the process we use to standardize the stratified preoperative evaluation of patients scheduled for ambulatory surgery.

A Screening Tool to Quickly Identify Movement Disorders in Patients with Inborn Errors of Metabolism.

Movement disorders are frequent in patients with inborn errors of metabolism (IEMs) but poorly recognized, particularly by nonmovement disorder specialists. We propose an easy-to-use clinical screening tool to help recognize movement disorders. The aim is to develop a user-friendly rapid screening tool for nonmovement disorder specialists to detect moderate and severe movement disorders in patients aged ≥4 years with IEMs. Videos of 55 patients with different IEMs were scored by experienced movement disorder specialists (n=12). Inter-rater agreements were determined on the presence and subtype of the movement disorder. Based on ranking and consensus, items were chosen to be incorporated into the screening tool. A movement disorder was rated as present in 80% of the patients, with a moderate inter-rater agreement (κ =0.420, P< 0.001) on the presence of a movement disorder. When considering only moderate and severe movement disorders, the inter-rater agreement increased to almost perfect (κ=0.900, P< 0.001). Dystonia was most frequently scored (27.3%) as the dominant phenotype. Treatment was mainly suggested for patients with moderate or severe movement disorders. Walking, observations of the arms, and drawing a spiral were found to be the most informative tasks and were included in the screening tool. We designed a screening tool to recognize movement disorders in patients with IEMs. We propose that this screening tool can contribute to select patients who should be referred to a movement disorder specialist for further evaluation and, if necessary, treatment of the movement disorder. © 2023 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Hyperkalemia Detection in Emergency Departments Using Initial ECGs: A Smartphone AI ECG Analyzer vs. Board-Certified Physicians.

Hyperkalemia is a potentially fatal condition that mandates rapid identification in emergency departments (EDs). Although a 12-lead electrocardiogram (ECG) can indicate hyperkalemia, subtle changes in the ECG often pose detection challenges. An artificial intelligence application that accurately assesses hyperkalemia risk from ECGs could revolutionize patient screening and treatment. We aimed to evaluate the efficacy and reliability of a smartphone application, which utilizes camera-captured ECG images, in quantifying hyperkalemia risk compared to human experts. We performed a retrospective analysis of ED hyperkalemic patients (serum potassium ≥ 6 mmol/L) and their age- and sex-matched non-hyperkalemic controls. The application was tested by five users and its performance was compared to five board-certified emergency physicians (EPs). Our study included 125 patients. The area under the curve (AUC)-receiver operating characteristic of the application's output was nearly identical among the users, ranging from 0.898 to 0.904 (median: 0.902), indicating almost perfect interrater agreement (Fleiss' kappa 0.948). The application demonstrated high sensitivity (0.797), specificity (0.934), negative predictive value (NPV) (0.815), and positive predictive value (PPV) (0.927). In contrast, the EPs showed moderate interrater agreement (Fleiss' kappa 0.551), and their consensus score had a significantly lower AUC of 0.662. The physicians' consensus demonstrated a sensitivity of 0.203, specificity of 0.934, NPV of 0.527, and PPV of 0.765. Notably, this performance difference remained significant regardless of patients' sex and age (P < 0.001 for both). Our findings suggest that a smartphone application can accurately and reliably quantify hyperkalemia risk using initial ECGs in the ED.

Vocal Fold Pathologies Among Undergraduate Singing Students In Three Different Genres.

This study aimed to compare the prevalence and incidence of vocal fold pathologies among undergraduate classical, musical theatre, and contemporary commercial music (CCM) students over two-time points. This study is part of a longitudinal investigation. Videostroboscopic examinations were rated, with consensus among three of four expert blinded raters confirming the presence of pathology. Association between genre of singer and the presence of pathology, interrater reliability, and intra-rater reliability were calculated. Prevalence and incidence of pathologies were compared across genres. During first-year evaluations, 32% of musical theatre, 18% of CCM, and 0% of classical students had vocal pathologies. The prevalence at third-year evaluations showed 22% of classical, 39% of musical theatre, and 27% of CCM participants having vocal fold pathologies. The incidence of pathologies was 67% of musical theatre students compared to 22% of classical students and 27% of CCM students. The four raters demonstrated fair to moderate interrater agreement. Singing Voice Handicap Index-10 scores were normal for CCM singers at both time points but elevated for musical theatre and classical singers. No classical singers were found to have pathology during first-year evaluations, although CCM and musical theatre singers showed evidence of vocal fold pathologies. At third-year evaluations, all three genres had an apparent increase in prevalence of pathologies. Implications of this study suggest that more time in the field and intense voice usage may lead to a greater risk of pathology for all singers, regardless of genre. 2 Laryngoscope, 133:2317-2324, 2023.

Reliability of Radiographic Imaging Characteristics for Osteochondritis Dissecans of the Capitellum

Background: A primary challenge in the treatment of capitellar osteochondritis dissecans (OCD) is accurate imaging assessment. Radiographic classification consensus is not available in the current literature, and correlation of radiographs with lesion stability and resultant best treatment is lacking. Purpose: To determine the inter- and intrarater reliability of the presence or absence and common radiographic characteristics of capitellar OCD lesions. Study Design: Cohort study (Diagnosis); Level of evidence, 3. Methods: Anteroposterior, lateral, and oblique radiographs for 29 cases were reviewed by 7 orthopaedic surgeons. Images were assessed for elbow anthropometry and morphology, OCD presence, lesion characteristics, the presence of progeny bone and progeny features, and radial head abnormalities. Intra- and interrater reliability was assessed using Fleiss and Cohen kappa for nominal variables and intraclass correlation coefficients (ICCs) for continuous variables. Results: Surgeons demonstrated substantial to excellent inter- and intrarater reliability when assessing elbow characteristics: anthropometric (interrater ICC, 0.94-0.99; intrarater ICC, 0.82-0.96) and morphologic (Fleiss, 0.61-0.76; Cohen, 0.68). When the OCD lesion was assessed, fair to moderate interrater agreement was found for classifying the absence or presence of a lesion (Fleiss, 0.28-0.46) and the location of the OCD (Fleiss, 0.24-0.52), poor agreement for assessing the contour of the lesion (Fleiss, 0.00-0.09), and excellent agreement for measuring the size of the lesion (ICC, 0.82-0.94). Poor to fair interrater agreement was found for radial head abnormalities (Fleiss, 0.00-0.27). Progeny bone visualization and fragmentation demonstrated moderate interrater agreement (Fleiss, 0.43-0.47) where displacement of the bone demonstrated poor interrater agreement (Fleiss, 0.11-0.16). Intrarater agreement for OCD lesion characteristics, progeny bone visualization, and progeny bone features was moderate to excellent. Conclusion: Given only the fair to moderate agreement among raters for identifying OCD on radiographs, this imaging modality may not serve as a dependable screening tool in isolation. Additional imaging should be obtained if the clinical presentation suggests capitellar OCD and a definitive diagnosis is not possible with radiographs. However, clinicians can reliability measure the size of radiographically apparent OCD, suggesting that radiographs may serve as an appropriate imaging modality for follow-up care.

Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies.

Here we describe the development and evaluation of a survey instrument to assess the research suitability of veterinary electronic medical records (EMRs) through the conduct of two studies as part of the Dog Aging Project (DAP). In study 1, four reviewers used the instrument to score a total of 218 records in an overlapping matrix of pairs to assess inter-rater agreement with respect to appropriate format (qualification), identification match (verification), and record quality. Based upon the moderate inter-rater agreement with respect to verification and the relatively large number of records that were incorrectly rejected the instrument was modified and more specific instructions were provided. In study 2, a modified instrument was again completed by four reviewers to score 100 different EMRs. The survey scores were compared to a gold standard of board-certified specialist review to determine receiver operating curve statistics. The refined survey had substantial inter-rater agreement across most qualification and verification questions. The cut-off value identified had a sensitivity of 95 and 96% (by reviewer 1 and reviewer 2, respectively) and a specificity of 82% and 91% (by reviewer 1 and reviewer 2, respectively) to predict gold standard acceptance or rejection of the record. Using just qualification and verification questions within the instrument (as opposed to full scoring) minimally impacted sensitivity and specificity and resulted in substantial time savings in the review process.

Two-type classification system for femoral trochlear dysplasia in recurrent patellar instability based on three-dimensional morphology.

Radiographic and two-dimensional (2D) CT/MRI analysis of femoral trochlear dysplasia play a significant role in surgical decision-making for recurrent patellar instability. However, the three-dimensional morphology of dysplastic trochlea is rarely studied due to the limitations of conventional imaging modalities. This study aimed to (1) develop a 3D morphological classification for trochlear dysplasia based on the concavity of the trochlear groove and (2) analyze the interrater reliability of the classification system. The 3D trochleae of 132 knees with trochlear dysplasia and recurrent patellar instability were reconstructed using CT scan data and classified using the innovative classification criteria between January 2016 and June 2020. A concave trochlear sulcus with sloped medial and lateral trochlear facets was classified as Type I trochlea. The trochlear groove with no concavity is classified as Type II. Furthermore, in Type II, the trochlea with the elevated trochlear floor at the proximal part was identified as IIa and the trochlea with the hypoplastic trochlear facets as IIb. The intra- and inter-rater reliability was examined using kappa (κ) statistics. The 3D classification system showed substantial intra-rater agreement and moderate interrater agreement (0.581 ~ 0.772). The intra- and interrater agreement of Dejour's four-grade classification was fair-to-moderate (0.332 ~ 0.633). Eighty-one trochleae with concave trochlear sulcus were classified as Type I, and fifty-one without concavity as Type II. Twenty-five non-concave trochleae were classified as type IIa due to the elevated trochlear floor and 26 trochleae into IIb with the hypoplasia of trochlear facets. This study developed a 3D classification system to classify trochlear dysplasia according to trochlear concavity and morphology of the trochlear facets. On CT/MRI scans or 3D reconstructions, the interpretation of features of dysplastic trochleae may vary, especially for the flat and convex trochleae. The novel system provides morphological evidence for when to consider trochleoplasty according to the different types of trochlear sulcus.