Prevalence-adjusted Bias-adjusted Kappa Research Articles

High prevalence of radiographic erosions in early, untreated PsA: results from the SpARRO cohort

AimsTo investigate the prevalence and distribution of bone erosions in an early psoriatic arthritis (PsA) population using conventional radiography (CR) and to explore the agreement between CR and ultrasound (US)...

Agreement in the Postural Assessment of Older Adults by Physical Therapists Using Clinical and Imaging Methods.

Postural assessment is one of the indicators of health status in older adults. Since the number of older adults is on the rise, it is essential to assess simpler methods and automated ones in the future. Therefore, we focused on a visual method (imaging method). The purpose of this study is to determine the degree of agreement between the imaging method and the palpation and visual methods (clinical method). In addition, the influence of differences in the information content of the sagittal plane images on the assessment was also investigated. In this experiment, 28 sagittal photographs of older adults whose posture had already been assessed using the clinical method were used. Furthermore, based on these photographs, 28 gray and silhouette images (G and S images) were generated, respectively. The G and S images were assessed by 28 physical therapists (PTs) using the imaging method. The assessment was based on the Kendall classification, with one of four categories selected for each image: ideal, kyphosis lordosis, sway back, and flat back. Cross-tabulation matrices of the assessments using the clinical method and imaging method were created. In this table, four categories and two categories of ideal and non-ideal (KL, SB, and FB) were created. The agreement was evaluated using the prevalence-adjusted bias-adjusted kappa (PABAK). In addition, sensitivity and specificity were calculated to confirm the reliability. When comparing the clinical and imaging methods in the four posture categories, the PABAK values were -0.14 and -0.29 for the S and G images, respectively. In the case of the two categories, the PABAK values were 0.57 and 0.5 for the S and G images, respectively. The sensitivity and specificity were 86% and 57% for the S images and 76% and 71% for the G images, respectively. The four categories show that the imaging method is difficult to assess regardless of the image processing. However, in the case of the two categories, the same assessment of the clinical method applied to the imaging method for both the S and G images. Therefore, no differences in image processing were observed, suggesting that PTs can identify posture using the visual method.

Relationship between animal-based on-farm indicators and meat inspection data in pigs

BackgroundThis study aimed to validate slaughterhouse indicators collected during meat inspection as an alternative to on-farm animal welfare indicators. For this purpose, the assessments of twelve on-farm and seven slaughterhouse indicators of 628 pigs from three different farms were combined into three indices, differentiated between on-farm and slaughterhouse: (1) limb health, (2) other organ health, and (3) respiratory health. At first, an assessment at animal-level using agreement parameters was carried out to ascertain whether the same welfare or health issues were identified on-farm and at slaughterhouse, taking the production period (farrowing, rearing and fattening period) and the last weeks before slaughtering into account. Second, the connection of slaughterhouse findings on the individual on-farm health indices was examined using logistic regressions, to determine whether certain welfare issues can be better monitored using slaughterhouse indicators.ResultsAcceptable agreement was determined using the Prevalence-Adjusted Bias-Adjusted Kappa (PABAK) for the farrowing and fattening period, but not for the rearing period. A more detailed analysis of the weeks before slaughter shows that there is still a poor agreement 8 weeks before slaughter and an acceptable agreement 4 weeks before slaughter. This indicated the slaughterhouse indicators pneumonia, pleuritis and pericarditis as possible estimators of fever and deviant behavior on-farm and the slaughterhouse indicators bursitis and joint inflammations as possible estimators of lameness. In the second part of the analysis, the connection of slaughterhouse findings on the individual on-farm health indices was investigated; a significant influence of the farm on the limb and respiratory indices and no significant influence of the slaughterhouse findings could be determined, provided that all weekly assessments during the lifetime of the pigs have been taken into account. However, an influence of the slaughterhouse findings on the respiratory index and on the other organ index could be determined if only the weekly assessments four and eight weeks before slaughter, respectively, were taken into account.ConclusionsIn general, the possible suitable indicators detected by the PABAK, could replace some health-related indicators but a complete substitution of on-farm welfare assessment is not possible. In addition, the traceability over time must be investigated further.

Validity of Partner Reports of Recent Condomless Sex.

Prostate-specific antigen (PSA), a biomarker of vaginal semen exposure, is less susceptible to bias than self-reported condom use behaviors. We examined the agreement of self-reported recent condomless sex (RCS) within couples and how these reports related to PSA detection. We analyzed data from a study conducted in Vietnam, 2017 to 2020, of 500 different-sex couples using condoms and no other contraceptive method to prevent pregnancy for 6 months. We assessed enrollment and 6-month data from vaginal swabs and questionnaires from both partners. We calculated Prevalence-Adjusted Bias-Adjusted Kappa (PABAK) to evaluate agreement of men's and women's reports. Among couples with detected PSA, we assessed partner concordance of RCS reporting. At enrollment (n = 499), 79.8% of couples reported no RCS, 16.4% reported RCS, and 3.8% had partner-discordant reports (PABAK, 0.93; 95% confidence interval, 0.91-0.97). At 6 months (n = 472), 91.7% reported no RCS, 5.7% reported RCS, and 2.5% had partner-discordant reports (PABAK, 0.98; 95% confidence interval, 0.96-1.0). Among couples with detected PSA at baseline (11%, n = 55), 36% reported no RCS, 55% reported RCS, and 6% had discordant reports; at 6 months (6.6%, n = 31), 58% reported no RCS, 35% reported RCS, and 3% had discordant reports. We observed high agreement regarding condomless sex within couples in a population using condoms as contraception in Vietnam; however, a high proportion of couples with detected PSA had both partners reporting no RCS, indicating that concordant reporting of no RCS does not indicate lack of semen exposure.

Can women's reports in client exit interviews be used to measure and track progress of antenatal care services quality? Evidence from a facility assessment census in Malawi.

Unlike household surveys, client exit interviews are conducted immediately after a consultation and therefore provides an opportunity to capture routine performance and level of service quality. This study examines the validity and reliability of women's reports on selected ANC interventions in exit interviews conducted in Malawi. Using data from the 2013-2014 Malawi service provision facility census, we compared women's reports in exit interviews regarding the contents of ANC received with reports obtained through direct observation by a trained healthcare professional. The validity of six indicators was tested using two measures: the area under the receiver operating characteristic curve (AUC), and the inflation factor (IF). Reliability of women's reports was measured using the Kappa coefficient (κ) and the prevalence-adjusted bias-adjusted kappa (PABAK). Finally, we examined whether reporting reliability varied significantly by individual and facility characteristics. Of the six indicators, two concrete and observable measures had high reporting accuracy and met the validity criteria for both AUC ≥ 0.7 and 0.75>IF>1.25, namely whether the provider prescribed or gave malaria prophylaxis (AUC: 0.84, 95% CI: 0.83-0.86; IF: 0.96) or iron/folic tablets (AUC: (0.84 95% CI: 0.81-0.87; IF:1.00). Whereas four measures related to counselling had lower reporting accuracy: whether the provider offered counselling about nutrition in pregnancy (AUC: 0.69, 95%CI: 0.67-0.71; IF = 1.26), delivery preparation (AUC: 0.62, 95% CI: 0.60-065; IF = 0.99), pregnancy related complications (AUC: 0.59, 95%CI: 0.56-0.61; IF = 1.11), and iron/folic acid side effects (AUC:0.58, 95% CI: 0.55-0.60; IF = 1.42). Similarly, the observable measures had high reliability with both κ and PABAK values in the ranges of ≥ 0.61 and ≥ 0.80. Respondent's age, primiparous status, number of antenatal visits, and the type of health provider increased the likelihood of reporting reliability. In order to enhance the measurement of quality of ANC services, our study emphasizes the importance of carefully considering the type of information women are asked to recall and the timing of the interviews. While household survey programmes such as the demographic health survey and multiple indicator cluster survey are commonly used as data sources for measuring intervention coverage and quality, policy makers should complement such data with more reliable sources like routine data from health information systems.

Patient-defined flares and disease activity worsening in 222 patients with psoriatic arthritis from 14 countries.

To explore patient-defined flares in psoriatic arthritis (PsA), compared to an increase in disease activity in psoriatic arthritis (DAPSA) and to analyze the validity of a patient-reported flare question. ReFlap (NCT03119805) was a longitudinal study in 14 countries of consecutive patients with definite PsA. Patients were seen twice in the context of usual care, 4.5±2.2 months apart. Flares were reported by patients and physicians at the second visit using a single question. DAPSA worsening was defined as a change to a higher DAPSA category. Agreement between the definitions of worsening was calculated by prevalence adjusted bias adjusted kappa (PABAK). Validity of patient-reported flare was assessed by comparing patients with versus without flare and transition to flares. In 222 patients, mean disease duration 10.8±8.3 years, 127 (58.8%) males: disease activity was low (mean DAPSA 11.5±14.0); 63.3% received a bDMARD. Patient-reported flares between the 2 visits were seen in 27% patients (for these patients, mean 2.2±3.7 flares per patient, mean duration 12.6±21.0 days per flare). Physician- reported flares were seen in 17.6% and worsening in DAPSA in 40.1% of patients. Agreement between definitions was moderate (PABAK=0.32-0.59). Patients in flare had significantly more active disease than patients not in flare for all outcomes (all P<0.001). At the patient-level, transition to flare state was associated to a worsening in disease activity and impact outcomes. Patient flares were frequent and were associated with active and symptomatic disease. These findings provide preliminary validation for patient-reported flares in PsA.

Influence of Patient and Clinician Gender on Emergency Department HEART Scores: A Secondary Analysis of a Prospective Observational Trial

Clinical decision aids can decrease health care disparities. However, many clinical decision aids contain subjective variables that may introduce clinician bias. The HEART score is a clinical decision aid that estimates emergency department (ED) patients' cardiac risk. We sought to explore patient and clinician gender's influence on HEART scores. In this secondary analysis of a prospective observational trial, we examined a convenience sample of adult ED patients at one institution presenting with acute coronary syndrome symptoms. We compared ED clinician-generated HEART scores with researcher-generated HEART scores blinded to patient gender. The primary outcome was agreement between clinician and researcher HEART scores by patient gender overall and stratified by clinician gender. Analyses used difference-in-difference (DiD) for continuous score and prevalence-adjusted, bias-adjusted Kappa (PABAK) for binary (low versus moderate/high risk) score comparison. All 336 clinician-patient pairs from the original study were included. In total, 47% (158/336) of patients were women, and 52% (174/336) were treated by a woman clinician. The DiD between clinician and researcher HEART scores among men versus women patients was 0.24 (95% CI-0.01 to 0.48). Compared with researchers, men clinicians assigned a higher score to men versus women patients (DiD 0.51 [95% CI 0.16 to 0.87]), whereas women clinicians did not (DiD 0.00 [95% CI-0.33 to 0.33]). Agreement was the highest among women clinicians (PABAK 0.72; 95% CI 0.61 to 0.81) and lowest among men clinicians assessing men patients (PABAK 0.47; 95% CI 0.29 to 0.66). Patient and clinician gender may influence HEART scores. Researchers should strive to understand these influences in developing and implementing this and other clinical decision aids.

Evaluating remote post-mortem veterinary meat inspections on pig carcasses using pre-recorded video material

BackgroundOfficial meat inspections at small-scale slaughterhouses and game-handling establishments in geographically remote areas place a heavy burden on the meat-producing food business operators. By performing meat inspections remotely using live-streamed video, instead of on-site, the official control could meet the goals of sustainability, resilience and logistics. We investigated the agreement between the two approaches at pig slaughter. Two official veterinarians (OVs) inspected 400 pig carcasses at a Swedish slaughterhouse, with each pig being inspected on site by one OV and remotely by the other. After a period of 3 to 6 months, video recordings of the remote inspections were assessed again by the same OVs, thus enabling direct comparisons of previous on-site inspections and renewed video-based inspections within the same OV.ResultsAgreement across 22 finding codes was generally very high for both OVs. In all but one case (whether to fully condemn a carcass), for both OVs, Prevalence-Adjusted Bias-Adjusted kappa was well above 0.8, indicating ‘almost perfect agreement’.ConclusionsThis study supports earlier findings that reliable post-mortem inspections can be performed using video, and indicates higher agreement between remote and on-site inspections if the same OV performs both.

Validade relativa e reprodutibilidade de indicadores da OMS para avaliação da alimentação de crianças menores de dois anos

The study aimed to evaluate relative validity and reproducibility of seven WHO indicators of dietary practices in children aged 6-23.9 months. Data from probabilistic sample of children who used primary healthcare services in Rio de Janeiro, Brazil were collected using a 24h dietary recall (24HR) and a closed questionnaire (Q1) on feeding in the day before the study. The last one was reapplied (Q2) around 16 days later. Validity was assessed by comparing the prevalence rates estimated by 24HR and Q1 and calculating the positive (PPV) and negative (NPV) predictive values, sensitivity (Se), specificity (Sp), and accuracy index (AI) for the resulting indicators. For reproducibility, estimated prevalence rates based on Q1 and Q2 were compared and the kappa index and prevalence-adjusted bias-adjusted kappa were estimated. Of the seven estimated indicators, the prevalence of two was overestimated (Continued breastfeeding: 50.0% vs 40.0%; Sweet beverage consumption: 65.1% vs 52.7%) and the prevalence of one was underestimated (Zero vegetable or fruit consumption: 6.5% vs 18.1%). For most indicators, Se and PPV were higher than Sp and NPV. The prevalence rates determined with Q1 and Q2 were similar for 6 indicators. More than half showed good, very good or excellent agreement.

MP68-18 END TO END ASSESSMENT OF SUTURING EXPERTISE (EASE) IN VIRTUAL REALITY SURGICAL SIMULATION WITH ASSOCIATED CLINICAL OUTCOMES

You have accessJournal of UrologyCME1 Apr 2023MP68-18 END TO END ASSESSMENT OF SUTURING EXPERTISE (EASE) IN VIRTUAL REALITY SURGICAL SIMULATION WITH ASSOCIATED CLINICAL OUTCOMES Timothy N. Chu, Runzhuo Ma, Alvin Hui, Oscar Gomez, Cherine H. Yang, Istabraq S. Dalieh, Elyssa Y. Wong, Ahmed Ghazi, John W. Davis, Brian J. Miles, Clayton Lau, and Andrew J. Hung Timothy N. ChuTimothy N. Chu More articles by this author , Runzhuo MaRunzhuo Ma More articles by this author , Alvin HuiAlvin Hui More articles by this author , Oscar GomezOscar Gomez More articles by this author , Cherine H. YangCherine H. Yang More articles by this author , Istabraq S. DaliehIstabraq S. Dalieh More articles by this author , Elyssa Y. WongElyssa Y. Wong More articles by this author , Ahmed GhaziAhmed Ghazi More articles by this author , John W. DavisJohn W. Davis More articles by this author , Brian J. MilesBrian J. Miles More articles by this author , Clayton LauClayton Lau More articles by this author , and Andrew J. HungAndrew J. Hung More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003331.18AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Virtual reality (VR) provides a standardized setting for surgical skill assessment but currently lacks high-level multi-institutional evidence to validate the predictive value of VR. We have previously developed a granular suturing assessment tool (EASE), which can accurately distinguish surgeon experience in live surgery. This present study aims to further validate EASE in a VR environment and evaluate the utility of VR as a tool to predict clinical outcomes. METHODS: 21 surgeons in this prospective five-center study completed VR simulation suturing exercises on the Surgical Science™ Flex VR simulator (Fig 1A). 5 independent and blinded graders received standardized training and provided technical skill assessment for VR using EASE. EASE was adapted to be relevant for a simulated environment (EASE-VR). Inter-rater reliability was measured using prevalence-adjusted bias-adjusted kappa (PABAK) with agreement >0.6 in all graded subskills. Expert (>100 robotic cases) and trainee (<100 cases) scores were compared using generalized estimating equation. Correlations for expert surgeon EASE-VR scores and post-surgical clinical outcomes were calculated using Cox regression while controlling for patient factors (age, BMI, preoperative PSA, biopsy Gleason Score, pathological tumor stage). RESULTS: 7 trainees (median caseload 60, range 0-80) and 14 expert surgeons (1500, range 275-3000) participated in this study. In VR, experts had significantly better Needle Handling (hold anglep=0.048) and Needle Driving (wrist rotation p=0.014) scores compared to trainees. There were no statistically significant differences in all other domains (p>0.05). For expert surgeons, Needle Driving (wrist rotation) was not significantly associated with urinary continence recovery. Expert Needle Handling (hold angle) scores were found to have a significant positive association with urinary continence recovery (Hazard Ratio 2.11, 95% Confidence Interval 1.44-3.09, p<0.001) (Fig 1B). CONCLUSIONS: EASE-VR can distinguish surgical experience in VR simulation exercises. Our results suggest that expert surgeon EASE-VR scores have positive association with post-surgical clinical outcomes. Source of Funding: Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R01CA251579. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e960 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Timothy N. Chu More articles by this author Runzhuo Ma More articles by this author Alvin Hui More articles by this author Oscar Gomez More articles by this author Cherine H. Yang More articles by this author Istabraq S. Dalieh More articles by this author Elyssa Y. Wong More articles by this author Ahmed Ghazi More articles by this author John W. Davis More articles by this author Brian J. Miles More articles by this author Clayton Lau More articles by this author Andrew J. Hung More articles by this author Expand All Advertisement PDF downloadLoading ...

Automated Coding of Job Descriptions From a General Population Study: Overview of Existing Tools, Their Application and Comparison.

Automatic job coding tools were developed to reduce the laborious task of manually assigning job codes based on free-text job descriptions in census and survey data sources, including large occupational health studies. The objective of this study is to provide a case study of comparative performance of job coding and JEM (Job-Exposure Matrix)-assigned exposures agreement using existing coding tools. We compared three automatic job coding tools [AUTONOC, CASCOT (Computer-Assisted Structured Coding Tool), and LabourR], which were selected based on availability, coding of English free-text into coding systems closely related to the 1988 version of the International Standard Classification of Occupations (ISCO-88), and capability to perform batch coding. We used manually coded job histories from the AsiaLymph case-control study that were translated into English prior to auto-coding to assess their performance. We applied two general population JEMs to assess agreement at exposure level. Percent agreement and PABAK (Prevalence-Adjusted Bias-Adjusted Kappa) were used to compare the agreement of results from manual coders and automatic coding tools. The coding per cent agreement among the three tools ranged from 17.7 to 26.0% for exact matches at the most detailed 4-digit ISCO-88 level. The agreement was better at a more general level of job coding (e.g. 43.8-58.1% in 1-digit ISCO-88), and in exposure assignments (median values of PABAK coefficient ranging 0.69-0.78 across 12 JEM-assigned exposures). Based on our testing data, CASCOT was found to outperform others in terms of better agreement in both job coding (26% 4-digit agreement) and exposure assignment (median kappa 0.61). In this study, we observed that agreement on job coding was generally low for the three tools but noted a higher degree of agreement in assigned exposures. The results indicate the need for study-specific evaluations prior to their automatic use in general population studies, as well as improvements in the evaluated automatic coding tools.

Reliability and Diagnostic Accuracy of Radiography for the Diagnosis of Calcium Pyrophosphate Deposition: Performance of the Novel Definitions Developed by an International Multidisciplinary Working Group.

To assess the reliability and diagnostic accuracy of new radiographic imaging definitions developed by an international multidisciplinary working group for identification of calcium pyrophosphate deposition (CPPD). Patients with knee osteoarthritis scheduled for knee replacement were enrolled. Two radiologists and 2 rheumatologists twice assessed radiographic images for presence or absence of CPPD in menisci, hyaline cartilage, tendons, joint capsule, or synovial membrane, using the new definitions. In case of disagreement, a consensus decision was made and considered for the assessment of diagnostic performance. Histologic examination of postsurgical specimens under compensated polarized light microscopy was the reference standard. Prevalence-adjusted bias-adjusted kappa values were used to assess reliability, and diagnostic performance statistics were calculated. Sixty-seven patients were enrolled for the reliability study. The interobserver reliability was substantial in most of the assessed structures when considering all 4 readers (κ range 0.59-0.90), substantial to almost perfect among radiologists (κ range 0.70-0.91), and moderate to almost perfect among rheumatologists (κ range 0.46-0.88). The intraobserver reliability was substantial to almost perfect for all the observers (κ range 0.70-1). Fifty-one patients were included in the accuracy study. Radiography demonstrated an overall specificity of 92% for CPPD, but sensitivity remained low for all sites and for the overall diagnosis (54%). The new radiographic definitions of CPPD are highly specific against the gold standard of histologic diagnosis. When the described radiographic findings are present, these definitions allow for a definitive diagnosis of CPPD, rather than other calcium-containing crystal depositions; however, a negative radiographic finding does not exclude the diagnosis.

Reliability assessment of ultrasound muscle echogenicity in patients with rheumatic diseases: Results of a multicenter international web-based study.

To investigate the inter/intra-reliability of ultrasound (US) muscle echogenicity in patients with rheumatic diseases. Forty-two rheumatologists and 2 radiologists from 13 countries were asked to assess US muscle echogenicity of quadriceps muscle in 80 static images and 20 clips from 64 patients with different rheumatic diseases and 8 healthy subjects. Two visual scales were evaluated, a visual semi-quantitative scale (0-3) and a continuous quantitative measurement ("VAS echogenicity," 0-100). The same assessment was repeated to calculate intra-observer reliability. US muscle echogenicity was also calculated by an independent research assistant using a software for the analysis of scientific images (ImageJ). Inter and intra reliabilities were assessed by means of prevalence-adjusted bias-adjusted Kappa (PABAK), intraclass correlation coefficient (ICC) and correlations through Kendall's Tau and Pearson's Rho coefficients. The semi-quantitative scale showed a moderate inter-reliability [PABAK = 0.58 (0.57-0.59)] and a substantial intra-reliability [PABAK = 0.71 (0.68-0.73)]. The lowest inter and intra-reliability results were obtained for the intermediate grades (i.e., grade 1 and 2) of the semi-quantitative scale. "VAS echogenicity" showed a high reliability both in the inter-observer [ICC = 0.80 (0.75-0.85)] and intra-observer [ICC = 0.88 (0.88-0.89)] evaluations. A substantial association was found between the participants assessment of the semi-quantitative scale and "VAS echogenicity" [ICC = 0.52 (0.50-0.54)]. The correlation between these two visual scales and ImageJ analysis was high (tau = 0.76 and rho = 0.89, respectively). The results of this large, multicenter study highlighted the overall good inter and intra-reliability of the US assessment of muscle echogenicity in patients with different rheumatic diseases.

The effect of short-course point-of-care echocardiography training on the performance of medical interns in children.

Point-of-care ultrasound (POCUS) can add complementary information to physical examination. Despite its development in several medical specialties, there is a lack of similar studies on children by medical interns and cardiologists. Therefore, investigators aimed to assess the effect of short-course training on the performance of medical interns in point-of-care echocardiography in children. This analytic cross-sectional study was conducted on 161 hospitalized children in 17 Shahrivar children's hospital, Iran, from January 2021 to May 2021. Seven interns (trainees) participated in a short course of point-of-care echocardiography to assess left ventricular ejection fraction (LVEF), inferior vena cava collapsibility index (IVCCI), and the presence of pericardial effusion (PEff). Each patient underwent point-of-care echocardiography by one of the trainees. Then, in less than one hour, the echocardiography was performed by a single cardiologist. Agreement between the cardiologist and trainees was examined using Cohen's kappa coefficient and Prevalence-Adjusted Bias-Adjusted Kappa (PABAK). For numerical variables, the agreement was examined using the concordance correlation coefficient (CCC) and intraclass correlation coefficient (ICC). Results showed that the cardiologist and trainees detected LVEF >50, IVCCI >50%, and the absence of PEff in most of the participants. A good agreement in terms of ICC and CCC for LVEF (0.832 and 0.831, respectively) and a good agreement in terms of ICC and CCC for IVCCI (0.878 and 0.877, respectively) were noted. Using categorical scoring of LVEF and IVCCI showed 94.4% and 87.6% complete agreement, respectively. Furthermore, using categorical scoring of LVEF and IVCCI, Cohen's kappa coefficient was 0.542 (moderate) and 0.619 (substantial), respectively. The PABAK for LVEF and IVCCI were 0.886 (almost perfect) and 0.752 (substantial), respectively. For PEff, Cohen's kappa and PABAK were 0.797 (moderate) and 0.988 (almost perfect), respectively, and the complete agreement was noted in 160 patients (99.4%). This study showed that a short teaching course could help medical interns to assess LVEF, IVCCI, and PEff in children. Therefore, it seems that adding this course to medical interns' curricula can be promising.

An Assessment Tool to Provide Targeted Feedback to Robotic Surgical Trainees: Development and Validation of the End-To-End Assessment of Suturing Expertise (EASE).

To create a suturing skills assessment tool that comprehensively defines criteria around relevant sub-skills of suturing and to confirm its validity. 5 expert surgeons and an educational psychologist participated in a cognitive task analysis (CTA) to deconstruct robotic suturing into an exhaustive list of technical skill domains and sub-skill descriptions. Using the Delphi methodology, each CTA element was systematically reviewed by a multi-institutional panel of 16 surgical educators and implemented in the final product when content validity index (CVI) reached ≥0.80. In the subsequent validation phase, 3 blinded reviewers independently scored 8 training videos and 39 vesicourethral anastomoses (VUA) using EASE; 10 VUA were also scored using Robotic Anastomosis Competency Evaluation (RACE), a previously validated, but simplified suturing assessment tool. Inter-rater reliability was measured with intra-class correlation (ICC) for normally distributed values and prevalence-adjusted bias-adjusted Kappa (PABAK) for skewed distributions. Expert (≥100 prior robotic cases) and trainee (<100 cases) EASE scores from the non-training cases were compared using a generalized linear mixed model. After two rounds of Delphi process, panelists agreed on 7 domains, 18 sub-skills, and 57 detailed sub-skill descriptions with CVI ≥ 0.80. Inter-rater reliability was moderately high (ICC median: 0.69, range: 0.51-0.97; PABAK: 0.77, 0.62-0.97). Multiple EASE sub-skill scores were able to distinguish surgeon experience. The Spearman's rho correlation between overall EASE and RACE scores was 0.635 (p=0.003). Through a rigorous CTA and Delphi process, we have developed EASE, whose suturing sub-skills can distinguish surgeon experience while maintaining rater reliability.

Ultrasonography of Inflammatory and Structural Lesions in Hand Osteoarthritis: An Outcome Measures in Rheumatology Agreement and Reliability Study.

To standardize and assess the reliability of ultrasonographic assessment of inflammatory and structural lesions in patients with hand osteoarthritis (OA). The Outcome Measures in Rheumatology Ultrasound Working Group selected synovial hypertrophy (SH), joint effusion (JE), and power Doppler (PD) signals as the main inflammatory lesions in hand OA, and suggested osteophytes in the scapho-trapezio-trapezoid (STT) and cartilage defects in the proximal interphalangeal (PIP) joints as novel additions to previous structural scoring systems. A complementary imaging atlas provided detailed examples of the scores. A reliability exercise of static images was performed for the inflammatory features, followed by a patient-based exercise with 6 sonographers testing inflammatory and structural features in 12 hand OA patients. We used Cohen's kappa for intrareader and Light's kappa for interreader reliability for all features except PD, in which prevalence-adjusted bias-adjusted kappa (PABAK) was applied. Percentage agreement was also assessed. The web-based reliability exercise demonstrated substantial intra- and interreader reliability for all inflammatory features (κ > 0.64). In the patient-based exercise, intra- and interreader reliability, respectively, varied: SH κ=0.73 and 0.45; JE κ=0.70 and 0.55; PD PABAK=0.90 and 0.88; PIP joint cartilage κ=0.56 and 0.45; and STT osteophytes κ=0.62 and 0.36. Percentage close agreement was high for all features (>85%). With ultrasound, substantial to excellent intrareader reliability was found for inflammatory features of hand OA. Interreader reliability was moderate, but overall high close agreement between readers suggests that better reliability is achievable after further training. Assessment of osteophytes in the STT joint and cartilage in the PIP joints achieved less reliability and the latter is not endorsed.

Clinical Accuracy of the Lateral-Anterior Drawer Test for Diagnosing Posterior Cruciate Ligament Rupture

BackgroundCommonly used clinical posterior cruciate ligament (PCL) tests present with diagnostic weaknesses requiring alternative clinical tests. The Lateral-Anterior Drawer test (LAD-test) is a suggested alternative that previously demonstrated concurrent validity in situ. Further in vivo LAD-test clinical accuracy examination is required prior to any recommendation for clinical adoption. Thus, this case–control study aims to (1) investigate the LAD-test’s in vivo interrater and intra-rater reliability; (2) establish LAD-test concurrent validity against MRI as the reference standard; and (3) examine the correspondence between examiners’ professional working experience and LAD-test diagnostic accuracy.MethodsThree examiners with different professional experience levels, blindfolded during testing, and blinded from subjects’ identity, medical history, and reference test outcome performed all LAD-testing twice per subject. Reliability analyses included percent agreement, Fleiss’ kappa and Cohen’s kappa coefficients with 95% Confidence Intervals (CIs) and prevalence-adjusted bias-adjusted kappa (PABAK) calculations. Validation parameters included sensitivity, specificity, likelihood ratios (LR + ; LR-), and predictive values (PPV; NPV) each accompanied by 95%CIs; each tester’s percent agreement with the MRI; and their Youden Index.ResultsThe study sample was comprised of 31 subjects of which 14 had a history of unilateral full-thickness PCL-rupture. Their 14 contralateral knees and both knees of 17 healthy subjects served as controls. In vivo LAD-test performance did not produce any negative ramifications for the tested subjects. Interrater reliability was moderate (test-1: Fleiss’κ = 0.41; 95% CI 0.40;0.41; test-2:Fleiss’κ = 0.51; 95% CI 0.50;0.51). Pairwise examiner’s LAD-test outcome agreement ranged from 74 to 89%. Pairwise interrater reliability was fair-to-substantial (κ = 0.27 to κ = 0.65) with moderate-to-substantial PABAK (0.48–0.77). Intra-rater reliability was substantial-to-almost perfect (PABAK 0.65–0.97). Sensitivity and specificity ranged from 57 to 86% and 83 to 98%, respectively. The advanced and novice clinicians’ Youden Indexes were acceptable. The same examiners’ positive likelihood ratios revealed important and relative important effects, respectively. Positive predictive values were considerable for the advanced and novice clinicians, while negative predictive values were high for all examiners.ConclusionOverall, the study results suggested LAD-test practicability. In vivo LAD-test performance did not produce any negative ramifications for the tested subjects. In subjects presenting with a chronic PCL-deficiency (i.e., > 3 months since initial injury), the LAD-test’s clinical accuracy was comparable-to-superior to other commonly used clinical PCL-tests. Future studies to establish the LAD-test’s usefulness in isolation as well as in combination with other clinical tests for acute PCL-rupture diagnostics are warranted.Trial registration numberDRKS00013268 (09. November 2017).

Identifying Breast Cancer Recurrence in Administrative Data: Algorithm Development and Validation.

Breast cancer recurrence is an important outcome for patients and healthcare systems, but it is not routinely reported in cancer registries. We developed an algorithm to identify patients who experienced recurrence or a second case of primary breast cancer (combined as a “second breast cancer event”) using administrative data from the population of Ontario, Canada. A retrospective cohort study design was used including patients diagnosed with stage 0-III breast cancer in the Ontario Cancer Registry between 1 January 2009 and 31 December 2012 and alive six months post-diagnosis. We applied the algorithm to healthcare utilization data from six months post-diagnosis until death or 31 December 2013, whichever came first. We validated the algorithm’s diagnostic accuracy against a manual patient record review (n = 2245 patients). The algorithm had a sensitivity of 85%, a specificity of 94%, a positive predictive value of 67%, a negative predictive value of 98%, an accuracy of 93%, a kappa value of 71%, and a prevalence-adjusted bias-adjusted kappa value of 85%. The second breast cancer event rate was 16.5% according to the algorithm and 13.0% according to manual review. Our algorithm’s performance was comparable to previously published algorithms and is sufficient for healthcare system monitoring. Administrative data from a population can, therefore, be interpreted using new methods to identify new outcome measures.

Comparison of a chemiluminescence immunoassay and an enzyme immunoassay for detection of IgM antibodies against measles, mumps, rubella, cytomegalovirus (CMV), Epstein Barr virus (EBV), and human herpes virus (HHV) -1 and -2 infections

PurposeOutbreaks of vaccine-preventable viral diseases have been increasingly reported globally over the past few years. The burden of congenital viral infections, their impact on physical and mental development and the resulting economic loss to the family and the community are also well known. IgM antibody detection has been convenient in the diagnosis of acute viral infections, particularly in settings with limited resources where molecular tests are not feasible. MethodsThis is a comparative study between a chemiluminescence immunoassay (Liaison, DiaSorin, Saluggia, Italy) and an enzyme linked immunosorbent assay (ELISA) (Euroimmun, Lubeck, Germany) for the detection of IgM antibody against measles, mumps, rubella, CMV, EBV and HHV-1 and -2 viruses using a total of 345 samples. Results are expressed as agreement using kappa statistics. ResultsIn this study, CLIA is perfectly comparable to ELISA for the detection of IgM antibodies against measles (0.86) and mumps (0.92) with a moderate agreement for rubella (0.52), CMV (0.57), EBV (0.50), and HHV-1 and -2 (0.47) assays. However, a PABAK (prevalence-adjusted bias-adjusted kappa) showed improved agreement for rubella (0.64), CMV (0.65), EBV (0.60), and HHV-1 and -2 (0.88) assays. ConclusionsIgM antibody assays (CLIA and ELISA) against measles and mumps virus can be comparably used depending on the laboratory setup, throughput and expertise.

Detection of Anterior Hyaloid Membrane Detachment Using Deep-Range Anterior Segment Optical Coherence Tomography.

The visibility of anterior hyaloid membrane (AHM) and Berger’s space in phakic eyes was investigated. In 624 eyes of 624 patients, the retrolental space was scanned with the deep-range anterior segment optical coherence tomography (AS-OCT, CASIA2, Tomey). Subgroup analysis was conducted in 223 eyes undergoing cataract surgery. The logistic regression analysis using the backward-elimination method was performed to evaluate the influence of various factors on the visibility of AHM (dependent variable). Explanatory variables included age, gender, axial length, corneal power, corneal cylinder, and the Co III gradings. Intrarater repeatability for AHM visibility was excellent with the prevalence-adjusted bias-adjusted kappa (κ coefficient) of 0.90. AHM was observed in 43 eyes (6.9%). The patients with visible AHM (68.1 ± 8.8 years) were significantly older than those without visible AHM (52.6 ± 25.6 years) (p < 0.001). The logistic regression analysis in the cataract surgery subgroup revealed that axial length (p = 0.030) and corneal power (p = 0.043) were significantly associated with AHM visibility. The mean absolute refractive error from target was significantly larger in eyes with visible AHM (0.670 ± 0.384 D) than with invisible AHM (0.494 ± 0.412 D) (p = 0.037). The postoperative refractive prediction was less accurate in eyes with visible AHM, but no significant tendency existed in terms of myopic or hyperopic shifts.