Low Level Of Agreement Research Articles

(121) Utilizing ChatGPT for Urology Trainee Simulation of Peyronie’s Disease Counseling

Abstract Introduction Peyronie’s disease is a medical issue with significant psychological overtones. However, trainee exposure to initial counseling and management can be limited and therefore understanding both the disease state and its psychological effects can be difficult to master. Institutions have incorporated simulation-based education to augment resident skills and knowledge in topics such as transgender health and SICU patient management. The gold standard of assessing trainees in simulated clinical interaction has been objective structured clinical examinations (OSCE). However, novel AI-based technologies like ChatGPT offer a promising alternative for resident education and assessment. ChatGPT, developed by OpenAI, utilizes natural language processing technology to provide human-like responses to open-ended questions posed by users. Preliminary reports have already described its use in training emergency medicine residents in delivering bad news. Objective To simulate a realistic clinical scenario utilizing ChatGPT to improve resident and student confidence in Peyronie’s disease evaluation and treatment, and to assess ChatGPT’s ability to deliver actionable feedback to trainees on both knowledge base and patient interaction compared to evaluation by a sexual medicine specialist. Methods ChatGPT Plus was utilized to roleplay the patient in an initial consultation for Peyronie’s disease. A detailed input was composed to establish the rules of the simulation (Figure 1). ChatGPT was prompted to conduct a final assessment of trainee performance with standardized scales in accuracy, comprehensiveness, and empathy (A, C, E). ChatGPT was prompted to use the 2015 AUA Peyronie’s Disease Diagnosis and Treatment Guidelines when formulating its assessment and feedback. Three trainees with different experience levels performed the simulation. At completion, a fellowship-trained sexual medicine specialist separately graded each trainee’s performance using the same scale. Cohen’s kappa was calculated to assess inter-rater reliability. Results Three urology trainees performed the simulation: a 3rd year medical student, a PGY2 resident, and a PGY4 resident. Trainee performance scores are shown in Table 1. Mean ChatGPT ratings were 8.7, 8.0, and 9.0 (A, C, E), while andrologists ratings were 7.0, 4.7, and 6.3 (A, C, E). Progressive improvement was demonstrated as trainee experience increased. ChatGPT generally rated trainees at higher competency levels than andrologists. Cohen’s kappa was calculated to be -0.125, -0.125, and 0.0 (A, C, E), indicating an overall low level of agreement between ChatGPT and andrologists assessment of trainees. Conclusions Our findings demonstrate that ChatGPT has potential to be a successful tool for urology resident and medical student education on Peyronie’s disease counseling. However, there was disagreement in trainee ratings between ChatGPT and the sexual medicine specialist. ChatGPT provided trainees constructive, actionable feedback referencing current guidelines (Figure 2). When given defined parameters for a clinical scenario, ChatGPT can be effective in designing a realistic standardized patient simulation. However, ChatGPT has not appeared to master the competency and accuracy of that expected from a sexual medicine specialist. Tweaking the initial input can allow for testing competency in various alternative clinical vignettes. Further collection of real-world data is needed to better assess ChatGPT’s rating of the learner’s performance, ideally in conjunction with national/international guidelines. Disclosure Any of the authors act as a consultant, employee or shareholder of an industry for: Fellow, Posterity Health, Coloplast, Hims, Genentech.

Is the Corrected Carotid Flow Time a Clinically Acceptable Surrogate Hemodynamic Parameter for the Left Ventricular Ejection Time?

ObjectiveThe corrected left ventricular ejection time (cLVET) comprises the phase from aortic valve opening to aortic valve closure corrected for heart rate. As a surrogate measure for cLVET, the corrected carotid flow time (ccFT) has been proposed in previous research. The aim of this study was to assess the clinical agreement between cLVET and ccFT in a dynamic clinical setting. MethodsTwenty-five patients with severe aortic valve stenosis (AS) were selected for transcatheter aortic valve replacement (TAVR). The cLVET and ccFT were derived from the left ventricular outflow tract (LVOT) and the common carotid artery (CCA) respectively, using pulsed wave Doppler ultrasound. Bazett's (B) and Wodey's (W) equation were used to calculate cLVET and ccFT. Measurements were performed directly before (T1), and after (T2) TAVR. Correlation, Bland-Altman, and concordance analyses were performed. ResultsThe cLVET decreased from T1 to T2 (p<0.001), with a relative reduction of 11% (B) and 9% (W). The ccFT decreased (p<0.001) with a relative reduction of 12% (B) and 10% (W). The correlation between cLVET and ccFT was strong for B (ρ=0.74, p<0.001), and W (ρ=0.81, p<0.001). The bias was -39 ms (B) and -37 ms (W), and the upper and lower level of agreement were 19 ms and -98 ms (B) and 5 ms and -78 ms (W), respectively. The trending ability between cLVET and ccFT was good (concordance 96%) for both B and W. ConclusionIn TAVR patients, the clinical agreement between cLVET and ccFT was acceptable, indicating that ccFT could serve as a surrogate measure for cLVET.

Managers’ attitudes as a critical success factor of kaizen

The purpose of our study is to examine the manager’s attitudes towards different aspects of kaizen, their commitment to the idea of kaizen, and their attitudes towards the motivation and participation of employees. We use a questionnaire to investigate the attitudes of 124 managers – 62 economists and 62 engineers, from 62 manufacturing companies operating in Serbia. We employ exploratory factor analysis (EFA) and Mann-Whitney’s U test. We find a moderate level of managers’ agreement with the statements on the importance and effects of kaizen, and a relatively low level of agreement with the statements on employees’ participation and motivation. Using the EFA, managers’ attitudes are grouped into three areas requiring attention – perceived kaizen effects, employees’ development and motivation, and employees’ participation. Finally, we find that economists and engineers do not differ in their attitudes towards kaizen, which indicates their similar and common attitudes towards certain aspects of kaizen. We contribute to the literature by examining managers’ attitudes towards the critical success factors of kaizen implementation and identifying the areas requiring more attention from the managers to support the social aspects of kaizen implementation and effects. Our results should change the perceptions of managers and contribute to the development of a new approach that involves more active and effective managers’ participation in the kaizen implementation. Our results can also be beneficial to practitioners seeking to implement and use kaizen more efficiently in their companies. Although our study does not consider the direct impact of a specific economic environment on kaizen implementation, it can be beneficial to companies in emerging and transitioning economies similar to Serbian.

Comparison of NIRS exercise intensity thresholds with maximal lactate steady state, critical power and rowing performance.

This study aimed to compare the intensity of deoxygenated haemoglobin concentration ([HHb]) and tissue saturation index (TSI) breakpoints ([HHb]-BP and TSI-BP) with maximal lactate steady state (MLSS) and critical power (CP), and to describe their association with 2000-m rowing ergometer performance. Fourteen male rowers performed on a rowing ergometer: I) a discontinuous incremental test with 3-min stages (INC3); II) a continuous incremental test with 1-min stages (INC1); III) constant workload tests to determine MLSS; and IV) performance tests of 500 m, 1000 m, 2000 m and 6000 m to determine CP.CP (257 ± 39 W; 3.79 ± 4.1 L · min-1) was higher than [HHb]-BP3 (205 ± 26 W; 3.48 ± 2.9 L · min-1), [HHb]-BP1 (207 ± 27 W; 3.27 ± 3.2 L · min-1), and TSI-BP3 (218 ± 31 W; 3.51 ± 3.0 L · min-1), but not higher than TSI-BP1 (222 ± 34 W; 3.43 ± 3.2 L · min-1). MLSS (187 ± 26 W; 3.33 ± 3.2 L · min-1) was lower than TSI-BP3 and TSI-BP1 for power output, but not different in any comparison for ⩒O2. The limits of agreement for power output and ⩒O2 suggest poor agreement among these thresholds. The low level of agreement compromises the use of [HHb]-BP and TSI-BP for estimating MLSS and CP; therefore, these thresholds should not be considered interchangeable.

Toward a Redefinition of Agricultural Drought Periods—A Case Study in a Mediterranean Semi-Arid Region

Drought is a powerful natural hazard that has significant effects on ecosystems amid the constant threats posed by climate change. This study investigates agricultural drought in a semi-arid Mediterranean basin through the interconnections among four indices: precipitation (meteorological reanalysis), vegetation development, thermal stress, and soil water deficit (remote sensing observations). While drought seems to be a clear concept with effective assessment tools (e.g., SPI and SPEI), the definition of drought periods is blurrier. This article examines the main drivers of agricultural drought, precipitation, soil moisture deficit, incipient vegetation development, and rising soil surface temperature. Their temporal connections in various agrosystems of the basin and the determination of drought periods by revisiting the run theory were investigated. The Pearson correlations at different spatial scales showed a medium to low level of agreement between the indices, which was explained by the geographical heterogeneity and the climatic variability between the agrosystems within the basin. It was also shown that the cascade of impacts expected from lower precipitations was revealed by the cross-correlation analysis. The connection between precipitation deficit and vegetation remains significant for at least one month for most pairs of indices, especially during drought events, suggesting that agricultural drought spells can be connected in time through the three or four selected indices. Short-, mid-, and long-term impacts of precipitation deficiencies on soil moisture, vegetation, and temperature were revealed. As expected, the more instantaneous variables of soil moisture and surface temperature showed no lag with precipitation. Vegetation anomalies at the monthly time step showed a two-month lag with a preceding effect of vegetation to precipitation. Finally, the determination of drought events and stages with varying thresholds on the run theory showed large variability in duration, magnitude, and intensity according to the choice of both normality and dryness thresholds.

In partnership for the goals? The level of agreement between SDG ratings

This study analyzes the level of agreement of Sustainable Development Goal (SDG) ratings across five different rating providers. It documents a low level of agreement that is particularly pronounced for companies from the Energy, Healthcare, and Basic Materials sectors. Moreover, the low level of agreement is mostly driven by some individual SDGs. When analyzing implications, we find different return characteristics and risk factor exposures of portfolios sorted according to SDG ratings of different rating providers. Overall, our analyses show that current SDG ratings fail to provide a clear signal of companies’ contribution to the SDGs which can have severe consequences for sustainability transitions and their financing.

Awareness of electronic cigarette industry practices and their associations with anti-electronic cigarette attitudes among susceptible US young adults.

Public education exposing cigarette industry practices have been effective in changing attitudes and preventing smoking among young people. It is unclear how much young adults are aware of e-cigarette industry practices, and how this awareness relates to anti-e-cigarette attitudes. We examined demographic correlates of awareness of e-cigarette industry practices and anti-e-cigarette attitudes, and the association between awareness of these practices with anti-e-cigarette attitudes. A US sample of young adults aged 18-30 years who do not use commercial tobacco products but are susceptible to e-cigarette use were cross-sectionally surveyed through online panel services from August 2021 to January 2022. Respondents reported their demographics, awareness of cigarette industry practices, awareness of e-cigarette industry practices and their level of agreement with four anti-e-cigarette attitude statements. We used multivariable linear regressions to examine demographic associations and the relationship between awareness of e-cigarette industry practices with each anti-e-cigarette attitude, adjusting for demographics and awareness of cigarette industry practices. Generally, Hispanic and Black young adults (vs White) and those with <US$75 000 annual household income (vs ≥US$75 000) knew of fewer e-cigarette industry practices. Black young adults (vs White) and those with <US$75 000 annual household income (vs ≥US$75,000) also had lower levels of agreement with anti-e-cigarette attitudes. Awareness of e-cigarette industry practices (vs awareness of zero practices) were associated with stronger agreement with each of the four anti-e-cigarette attitudes. Public education exposing e-cigarette industry practices may promote anti-e-cigarette attitudes among susceptible young adults who do not use commercial tobacco products. Future research should investigate the utility of anti-e-cigarette industry messaging.

Inter-rater reliability of the ‘Tool for assessing determinants of health in public space’ in a co-creative urban design process with care home residents in Barcelona: a Health CASCADE study

ABSTRACT This study aims to evaluate the inter-rater reliability of the Tool for assessing determinants of health in public space, a methodology developed to assess the potential health impact of public space improvement actions. The study involved a participatory evaluation conducted during an urban design co-creation process with older adults and researchers from various disciplines. Ten older adults from a care home in Barcelona and five researchers participated in the evaluation of five selected public spaces using the tool. An analysis using Intra-class Correlation Coefficient (ICC) was conducted to assess inter-rater reliability. A descriptive comparison of the results was performed based on the scores given to each determinant. The findings provide insights into the tool’s reliability and its potential for capturing diverse perspectives. The Tool does not appear to have high reliability when implemented in a community setting. The ICC values for most of the determinants are relatively low, indicating poor or moderate levels of agreement among the raters/judges. The study highlights the importance of involving end-users and researchers in the evaluation process, emphasizing the need for user-centered design and co-creation in urban planning. The results contribute to improving public health outcomes and promoting age-friendly environments in urban spaces.

Agreement of bioimpedance analysis and ultrasound scanning for fat mass, fat free mass and body fat percentage evaluation in the group of adult women

The study aims to perform an agreement analysis of bioimpedance (BIA) results obtained using АВС-02 “Medas” (Russia) and ultrasound scanning (US) using BodyMetrixTM (USA) for fat mass, fat free mass and body fat percentage in the group of females from Moscow. The study was performed with 180 female subjects 18-67 years of age. The agreement analysis conducted in the whole sample revealed a low level of agreement in estimating body fat percentage (CCC = 0.70 0.76 0.81) and fat free mass (CCC = 0.86 0.89 0.91), but agreement can be described as medium in estimating fat mass (CCC = 0.90 0.92 0.94). Then we adjusted the prediction equations and the agreement analysis was conducted again. Adjusted prediction equations improved the level of agreement to medium when estimating body fat percentage and fat free mass. Thus, the proposed equations can be used for the translation of body composition results obtained by US into the BIA data.

Influence of different time framings, reconstruction algorithms and post-processing methods on the quantification of myocardial blood flow from 13 N-NH3 PET images.

The aim was to investigate to what extent the quantification of myocardial blood flow (MBF) from dynamic 13 N-NH3 positron emission tomography(PET) images is affected by time frame schemes, time-of-flight (ToF), reconstruction algorithms, blood pool volume of interest (VOI) locations and compartment models in patients with suspected chronic coronary syndrome. A standard MBF value was determined from 25 patients' rest/stress 13 N-NH3 PET/CT images reconstructed with ordered subset expectation maximization (OSEM), 5 s time frame for the first frames without ToF, subsequently analyzed using a basal VOI and the deGrado compartment model. MBFs calculated using 2or 10 s for the first frames, ToF, block-sequential regularized expectation maximization (BSREM), apical or large VOI, Hutchins or Krivokapich compartment models were compared to MBFstandard in Bland-Altman plots (bias ± SD). Good agreement in global rest/stress MBF (mL/min/g) was found when changing the time frame scheme or reconstruction algorithm (MBFstandard vs. MBF2s : -0.02 ± 0.06; MBF10s : 0.01 ± 0.07; MBFBSREM : 0.01 ± 0.07), while a lower level of agreement was found when altering the other factors (MBFstandard vs.MBFToF : -0.07 ± 0.10; MBFapical VOI : -0.27 ± 0.25; MBFlarge VOI : -0.11 ± 0.10; MBFHutchins : -0.08 ± 0.10; MBFKrivokapich : -0.47 ± 0.50). Quantification of MBF from 13 N-NH3 PET images is more affected by choice of compartment models, ToF and blood pool VOIs than by different time frame schemes and reconstruction algorithms.

Diagnostic accuracy of handheld single-lead electrocardiographic devices to detect atrial fibrillation: an examination using more than 30,000 rhythm traces in a randomised control trial

Abstract Background Early atrial fibrillation (AF) diagnosis offers an opportunity to initiate management to reduce stroke risk. Hand-held single-lead electrocardiograph (ECG) devices are widely available, yet there is little data on the implementation of AF screening using the devices in high-AF-risk communities and examining the accuracy of the devices over time. Such data may provide insights into the possibility of using artificial intelligence to scale up screening. Purpose To examine the accuracy of AliveCor ECG devices’ automated interpretation against clinicians’ diagnoses and explore the characteristics of the ECGs of older community-dwelling people in a real-world setting. Methods This is an analysis of ECGs received from people aged ≥75 years participating in an implementation trial of a community AF screening program over 12 months. Clinicians reviewed all ECGs transmitted to a central portal. Clinicians’ interpretations were taken as diagnostically accurate. Participants and their general practitioners (GPs) were notified of AF. The devices’ automatic ECG interpretations, including the participants’ pulse rates automatically recorded by the devices, were extracted from the central portal and analysed using R v3.6.0. Results 200 participants were enrolled. From May 2021 to Feb 2023, 30,040 ECGs were received; of these traces, clinicians confirmed AF in 479 traces and no AF in 29,561 traces. The devices’ automatic algorithm had a sensitivity of 91.4 % (i.e., detected AF in 438 of these 479 traces) and specificity of 96.6% (no AF: 28,546 / 29,561 traces). In total, the devices’ automated interpretation identified 1,453 ECGs with "Possible AF", of which clinicians confirmed 911 (62.7%) were in sinus rhythm and 438 (30.1%) had AF (Table 1). Breaking down the statistics for various rhythms and comparing the devices’ automatic algorithms with the clinicians’ diagnoses, discrepancies were found in 5296 ECG traces (Kappa = 0.33). Of these, 4277 (80.8%) were interpreted by the device automated algorithm as "Unreadable" (interferences), "Unclassified" (the device was unable to determine a result), and "Too short" (&amp;lt; 30 seconds of recording). Excluding the 3799 "Unclassified’ traces, the level of agreement between the ECG devices and clinicians improved (Kappa 0.63). The device automatically reported "Unclassified" and "Possible AF" rhythm traces had wide dispersion of the participants’ pulse rates (Figure 1). Conclusions The ECG devices had high sensitivity to detect AF but a low level of agreement in differentiating various rhythms. The characteristics of the longitudinal ECG rhythm traces, including the participants’ pulse rates over time coupled with the clinicians’ diagnoses of this large dataset, could be used to improve the devices’ automated algorithm and inform artificial intelligence strategies for large-scale community AF screening programs.Table 1Figure 1

Intra- and inter-rater reliability of a manual codification system for footwear impressions: first lessons learned from the development of a footwear database for forensic intelligence purposes

To generate forensic intelligence from footwear impressions and link crime scenes, most law enforcement agencies and forensic laboratories rely on a manual codification system based on pattern recognition and classification by human analysts. However, although they are commonly used in practice, to date we still know little about the reliability of such systems. Taking advantage of the development of a footwear database for forensic intelligence purposes at the Laboratoire de sciences judiciaires et de médecine légale in Quebec (Canada), this study aims to make a preliminary assessment of the intra- and inter-rater reliability (i.e., the level of repeatability over time and the level of consensus between analysts) of the proposed codification system. To do so, three forensic intelligence analysts classified a set of 27 crime scene impressions and test impressions at two different times (two weeks apart). Percent agreement, Cohen’s Kappa, and Light’s Kappa were then calculated. Results show that two out of three analysts have reached an almost perfect level of intra-rater agreement, while the other have achieved a substantial level of intra-agreement, and that all analysts have reached a substantial level of inter-rater agreement. Findings suggest that, although a few patterns may have lower levels of agreement, overall, the developed codification system presents a satisfactory level of reliability. This preliminary study thus suggests that contrary to what advocates of fully automated systems may sometimes imply, manual codification of footwear impressions may be fairly appropriate for intelligence purposes. It calls for further evaluative research in the field.

A Digital Tool for the Self-Assessment of Homes to Increase Age-Friendliness: Validity Study.

Age-friendly environments in homes and communities play an important role in optimizing the health and well-being of society. Older people have strong preferences for remaining at home as they age. Home environment assessment tools that enable older people to assess their homes and prepare for aging in place may be beneficial. This study aims to establish the validity of a digital self-assessment tool by assessing it against the current gold standard, an occupational therapy home assessment. A cohort of adults aged ≥60 years living in metropolitan Adelaide, South Australia, Australia, assessed their homes using a digital self-assessment tool with 89 questions simultaneously with an occupational therapist. Adults who were living within their homes and did not have significant levels of disabilities were recruited. Cohen κ and Gwet AC1 were used to assess validity. A total of 61 participants (age: mean 71.2, SD 7.03 years) self-assessed their own homes using the digital self-assessment tool. The overall levels of agreement were high, supporting the validity of the tool in identifying potential hazards. Lower levels of agreement were found in the following domains: steps (77% agreement, Gwet AC1=0.56), toilets (56% agreement, κ=0.10), bathrooms (64% agreement, κ=0.46), and backyards (55% agreement, κ=0.24). Older people were able to self-assess their homes using a digital self-assessment tool. Digital health tools enable older people to start thinking about their future housing needs. Innovative tools that can identify problems and generate solutions may improve the age-friendliness of the home environment.

Repeatability and Agreement of Chord Mu Between Scheimpflug Tomography and Swept-source Optical Coherence Tomography.

To evaluate repeatability and agreemeny of chord mu between Scheimpflug tomography (Pentacam HR, Oculus Optikgeräte GmbH, Wetzlar, Germany) and swept-source optical coherence tomography-based optical biometer (IOLMaster 700, Carl Zeiss Meditec AG, Jena, Germany). In this retrospective study, 63 eyes of 33 patients were included. Chord mu, X, and Y Cartesian distances between the corneal vertex and the pupil center (Px and Py), and the pupil diameter were compared using two instruments. Repeatability was evaluated using intraclass correlation coefficient (ICC), coefficient of variation (CoV), and within-subject standard deviation (Sw). Inter-device agreement was evaluated using paired t-tests and Bland-Altman plots. Although Sw values for all parameters were similar between the two devices, CoV values of chord mu and pupil diameter were lower, and ICC values of those parameters were higher, in the IOLMaster 700 than in the Pentacam HR. Chord mu and pupil diameter values were higher in IOLMaster 700 than Pentacam HR (p < 0.01). The width of the 95% limit of agreement was wide for all parameters. IOLMaster 700 showed better repeatability than Pentacam HR in chord mu, Px, Py, and pupil diameter values. Because there were statistically significant differences and a low level of agreement in chord mu and pupil diameter values between the two devices, they cannot be used interchangeably.

Cross-validation of comorbidity items in two national databases in a sample of patients with end-stage kidney disease

BackgroundThe use of national medico-administrative databases for epidemiological studies has increased in the last decades. In France, the Healthcare Expenditures and Conditions Mapping (HECM) algorithm has been developed to analyse and monitor the morbidity and economic burden of 58 diseases. We aimed to assess the performance of the HECM in identifying different conditions in patients with end-stage kidney disease (ESKD) using data from the REIN registry (the French National Registry for patients with ESKD).MethodsWe included all patients over 18 years of age who started renal replacement therapy in France in 2018. Five conditions with a similar definition in both databases were included (ESKD, diabetes, human immunodeficiency virus [HIV], coronary insufficiency, and cancer). The performance of each SNDS algorithm was assessed using sensitivity, specificity, positive predictive values (PPVs), negative predictive values (NPVs), and Cohen’s kappa coefficient.ResultsIn total 5,971 patients were included. Among them, 81% were identified as having ESKD in both databases. Diabetes was the condition with the best performance, with a sensitivity, specificity, PPV, NPV, and Kappa coefficient all over 80%. Cancer had the lowest level of agreement with a Kappa coefficient of 51% and a high specificity and high NPV (94% and 95%). The conditions for which the definition in the HECM included disease-specific medications performed better in our study.ConclusionThe HECM showed good to very good concordance with the REIN database information overall, with the exception of cancer. Further validation of the HECM tool in other populations should be performed.

Contouring of emerging organs-at-risk (OARS) of the female pelvis and interobserver variability: a study by the Italian association of radiotherapy and clinical oncology (AIRO)

PurposeTo provide straightforward instructions for daily practice in delineating emerging organs-at-risk (OARs) of the female pelvis and to discuss the interobserver variability in a two-step multicenter study. Methods and materialsA contouring atlas with anatomical boundaries for each emerging OAR was realized by radiation oncologists and radiologists who are experts in pelvic imaging, as per their knowledge and clinical practice. These contours were identified as quality benchmarks for the analysis subsequently carried out. Radiation oncologists not involved in setting the custom-built contouring atlas and interested in the treatment of gynecological cancer were invited to participate in this 2-step trial. In the first step all participants were supplied with a selected clinical case of locally advanced cervical cancer and had to identify emerging OARs (Levator ani muscle; Puborectalis muscle; Internal anal sphincter; External anal sphincter; Bladder base and trigone; Bladder neck; Iliac Bone Marrow; Lower Pelvis Bone Marrow; Lumbosacral Bone Marrow) based on their own personal knowledge of pelvic anatomy and experience. The suggested OARs and the contouring process were then presented at a subsequent webinar meeting with a contouring laboratory. Finally, in the second step, after the webinar meeting, each participant who had joined the study but was not involved in setting the benchmark received the custom-built contouring atlas with anatomical boundaries and was requested to delineate again the OARs using the tool provided. The Dice Similarity Coefficient (DSC) and the Jaccard Similarity Coefficient (JSC) were used to evaluate the spatial overlap accuracy of the different volume delineations and compared with the benchmark; the Hausdorff distance (HD) and the mean distance to agreement (MDA) to explore the distance between contours. All the results were reported as sample mean and standard deviation (SD). ResultsFifteen radiation oncologists from different Institutions joined the study. The participants had a high agreement degree for pelvic bones sub-structures delineation according to DICE (IBM: 0.9 ± 0.02; LPBM: 0.91 ± 0.01). A moderate degree according to DICE was showed for ovaries (Right: 0.61 ± 0.16, Left: 0.72 ± 0.05), vagina (0.575 ± 0.13), bladder sub-structures (0.515 ± 0.08) and EAS (0.605 ± 0.05), whereas a low degree for the other sub-structures of the anal-rectal sphincter complex (LAM: 0.345 ± 0.07, PRM: 0.41 ± 0.10, and IAS: 0.4 ± 0.07). ConclusionThis study found a moderate to low level of agreement in the delineation of the female pelvis emerging OARs, with a high degree of variability among observers. The development of delineation tools should be encouraged to improve the routine contouring of these OARs and increase the quality and consistency of radiotherapy planning.

Interobserver agreement in the diagnosis of bladder outlet obstruction in women

The diagnosis of bladder outlet obstruction (BOO) in women is a challenge for functional urology. In Mexico there are few data that report the prevalence of OTSV in women, being up to 24% in a group of patients. The aim of this study is to compare six different definitions of bladder outlet obstruction and evaluate the interobserver agreement in an educational setting. Urodynamic studies (UDS) of women with and without diagnosis of BOO were retrospectively assesed. Farrar, Chassagne, Lemack, Defreitas, Blavais and Groutz, Solomon-Greenwell definitions were evaluated. All UDS were independently reviewed by 5 observers. The easiest, the hardest and the fastest were chosen. Interobserver agreement to classify the patients as obstructed was assessed by kappa reliability statistical analysis. We classified the type of mistakes the participants made; error of interpretation and miscalculation. A total of 28 urodynamic studies were reviewed. All observers had a substantial agreement (0.64-0.78) to classify BOO using all but Lemack and Solomon-Greenwell definitions. A total 120 errors from 840 responses were found; 45.8% errors of interpretation of UDS and 54.1% miscalculation of the equation. Finally, all the participants chose the Solomon-Greenwell was the most difficult definition. Chassagne, Defreitas and Farrar definitions proved substantial interobserver agreement. Solomon-Greenwell and Lemack´s definitions had the highest number of pitfalls and the lowest level of agreement.

Diffusion and Adoption of the ANA Nursing Scope and Standards of Practice Document: A National Survey

Professional organizations have a role in maintaining a healthy profession. Despite the significance of American Nurses Association (ANA) Nursing Scope and Standards of Practice document as a preeminent guide for nurses, little is known about its diffusion and adoption in nursing practice. Members of ANA Committee on Nursing Practice Standards developed and administered a national survey to generate new knowledge on diffusion characteristics of the document using Rogers’ diffusion of innovation model. A total of 1679 participants were included in the descriptive and content analysis. Despite high agreement levels among respondents regarding the purpose of and need for the document, findings revealed that the document was not widely diffused or adopted. Awareness, the first criteria for diffusion, was found to be both a barrier and a needed strategy. Observability, a key component for adoption, had a low agreement level for making a difference in one’s own and a peer’s practice.

A new software for automated counting of glistenings in intraocular lenses in vivo.

To assess the performance of a bespoke software for automated counting of intraocular lens (IOL) glistenings in slit-lamp images. IOL glistenings from slit-lamp-derived digital images were counted manually and automatically by the bespoke software. The images of one randomly selected eye from each of 34 participants were used as a training set to determine the threshold setting that gave the best agreement between manual and automatic grading. A second set of 63 images, selected using randomised stratified sampling from 290 images, were used for software validation. The images were obtained using a previously described protocol. Software-derived automated glistenings counts were compared to manual counts produced by three ophthalmologists. A threshold value of 140 was determined that minimised the total deviation in the number of glistenings for the 34 images in the training set. Using this threshold value, only slight agreement was found between automated software counts and manual expert counts for the validating set of 63 images (κ=0.104, 95%CI, 0.040-0.168). Ten images (15.9%) had glistenings counts that agreed between the software and manual counting. There were 49 images (77.8%) where the software overestimated the number of glistenings. The low levels of agreement show between an initial release of software used to automatically count glistenings in in vivo slit-lamp images and manual counting indicates that this is a non-trivial application. Iterative improvement involving a dialogue between software developers and experienced ophthalmologists is required to optimise agreement. The results suggest that validation of software is necessary for studies involving semi-automatic evaluation of glistenings.

Novices cannot fill the examiners’ shoes: Evidence of footwear examiners’ expertise in shoe comparisons

The value of a footwear examiner’s opinion centres on their ability to determine whether a particular shoe made an impression with greater accuracy than a novice. However, there has been limited research on the expertise of footwear examiners and the accuracy and reproducibility of their decisions. In the current study, we measured the accuracy and consensus of 31 footwear examiners versus a comparison group of 29 novices. Participants completed 20 ground truth known mock shoe comparisons. Results demonstrated that footwear examiners were more accurate than novices, regardless of comparison difficulty. Overall, on trials where probative decisions were given, examiners made false identifications and false exclusions on a total of 3% and 2% of trials, while novices made false identifications and false exclusions on a total of 19% and 17% of trials. Examiners also demonstrated better consensus in their opinions than novices, although both groups demonstrated low levels of agreement in their responses and variability in their interpretation of the conclusion scale. In summary, these findings support the proposition that footwear examiners show expert-level performance in matching known and unknown footwear impressions. These performance estimates may help the criminal justice system to appropriately value footwear examination evidence.