Inter-expert Agreement Research Articles

Evaluating the consistency of judgments derived through both in silico and expert application of the Cramer classification scheme

The Cramer classification scheme has emerged as one of the most extensively-adopted predictive toxicology tools, owing in part to its employment for chemical categorisation within threshold of toxicological concern evaluation. The characteristics of several of its rules have contributed to inconsistencies with respect to degree of hazard attributed to common (particularly food-relevant) substances. This investigation examines these discrepancies, and their origins, raising awareness of such issues amongst users seeking to apply and/or adapt the rule-set.A dataset of over 3000 compounds was assembled, each with Cramer class assignments issued by up to four groups of industry and academic experts. These were complemented by corresponding outputs from in silico implementations of the scheme present within Toxtree and OECD QSAR Toolbox software, including a working of a “Revised Cramer Decision Tree”. Consistency between judgments was assessed, revealing that although the extent of inter-expert agreement was very high (≥97%), general concordance between expert and in silico calls was more modest (∼70%). In particular, 22 chemical groupings were identified to serve as prominent sources of disagreement, the origins of which could be attributed either to differences in subjective interpretation, to software coding anomalies, or to reforms introduced by authors of the revised rules.

Refining sleep staging accuracy: transfer learning coupled with scorability models.

This study aimed to (1) improve sleep staging accuracy through transfer learning (TL), to achieve or exceed human inter-expert agreement and (2) introduce a scorability model to assess the quality and trustworthiness of automated sleep staging. A deep neural network (base model) was trained on a large multi-site polysomnography (PSG) dataset from the United States. TL was used to calibrate the model to a reduced montage and limited samples from the Korean Genome and Epidemiology Study (KoGES) dataset. Model performance was compared to inter-expert reliability among three human experts. A scorability assessment was developed to predict the agreement between the model and human experts. Initial sleep staging by the base model showed lower agreement with experts (κ = 0.55) compared to the inter-expert agreement (κ = 0.62). Calibration with 324 randomly sampled training cases matched expert agreement levels. Further targeted sampling improved performance, with models exceeding inter-expert agreement (κ = 0.70). The scorability assessment, combining biosignal quality and model confidence features, predicted model-expert agreement moderately well (R² = 0.42). Recordings with higher scorability scores demonstrated greater model-expert agreement than inter-expert agreement. Even with lower scorability scores, model performance was comparable to inter-expert agreement. Fine-tuning a pretrained neural network through targeted TL significantly enhances sleep staging performance for an atypical montage, achieving and surpassing human expert agreement levels. The introduction of a scorability assessment provides a robust measure of reliability, ensuring quality control and enhancing the practical application of the system before deployment. This approach marks an important advancement in automated sleep analysis, demonstrating the potential for AI to exceed human performance in clinical settings.

Rearranging Voice Disorders: Refining the New Two-Dimensional Continuous Model

BackgroundVoice and laryngeal pathologies are traditionally arranged in categories. A recent study has suggested an alternative approach, using two continuous scales, Organicity and Tonicity, which form a two-dimensional plane on which all pathologies/conditions can be arranged. ObjectivesThis study was designed to examine the validity and reliability of the new continuous 2D model and to learn how it is affected by experts' background characteristics. MethodsNinety-three international experts from 16 countries participated in the study and rated a comprehensive list of 35 laryngeal and voice pathologies/conditions on two continuous scales. On the Organicity scale, 0 represented 'non-organic', and 10 defined 'organic.' Similarly, on the Tonicity scale, 0 described 'hypotonic' and 10 represented 'hypertonic'. ResultsCronbach's alphas were high for Organicity and Tonicity (.99), with varied inter-expert agreement scores. Pathologies/conditions populated all four quadrants of the constructed two-dimensional plane, with a majority of 21 of the 35 located in the first quadrant (high-Tonicity, high-Organicity). Results showed strong replicability when compared to the preliminary study. In addition, ratings on the two scales were found consistent and statistically unaffected by the experts' background characteristics. ConclusionsThe new two-dimensional model is valid and reliable, and it provides a simple yet comprehensive approach for arranging voice disorders using a continuous perspective rather than a categorical one. Furthermore, the new model offers a framework that facilitates examination of the differences in how professional experts view different laryngeal pathologies and conditions.

Разработка краткой формы Шкалы оценки родителями одаренности детей (Short-Form Parent Rating Scale, SFPRS)

<p>The purpose of this paper was to develop a parent rating scale targeted at screening children in terms of giftedness. 292 parents of gifted and typical students participated in the study. Of the parents, 170 were female while 122 were male. In the study, parents were asked to indicate the prominent characteristics of their gifted children and 10 items were identified after the characteristics were examined by two experts. It was seen to that the items determined were compatible with the items in the parent rating scales in the literature. Content, construct and criterion-related validities were examined while the Cronbach alpha value as a sign of reliability was also examined. Content validity was approved by two experts, Kendall’s coefficient of concordance was calculated to determine inter-expert agreement and found to be 0.80. Confirmatory factor analysis results also supported a one-factor solution for an 8-item instrument. A comparison of scores given by parents of gifted and typical students also revealed a significant difference between two groups of the parents in favor of the parents having gifted children. The Cronbach’s alpha value was found to be 0.78, an acceptable value for internal consistency. Therefore, the Short-Form Parent Rating Scale (SFPRS) may be used to screen practically gifted children by the parents without using boring and long instruments. In addition, not checking convergent and divergent validity, test-retest reliability and measurement invariance are seen as the limitations of this paper.</p>

Random expert sampling for deep learning segmentation of acute ischemic stroke on non-contrast CT.

Outlining acutely infarcted tissue on non-contrast CT is a challenging task for which human inter-reader agreement is limited. We explored two different methods for training a supervised deep learning algorithm: one that used a segmentation defined by majority vote among experts and another that trained randomly on separate individual expert segmentations. The data set consisted of 260 non-contrast CT studies in 233 patients with acute ischemic stroke recruited from the multicenter DEFUSE 3 (Endovascular Therapy Following Imaging Evaluation for Ischemic Stroke 3) trial. Additional external validation was performed using 33 patients with matched stroke onset times from the University Hospital Lausanne. A benchmark U-Net was trained on the reference annotations of three experienced neuroradiologists to segment ischemic brain tissue using majority vote and random expert sampling training schemes. The median of volume, overlap, and distance segmentation metrics were determined for agreement in lesion segmentations between (1) three experts, (2) the majority model and each expert, and (3) the random model and each expert. The two sided Wilcoxon signed rank test was used to compare performances (1) to 2) and (1) to (3). We further compared volumes with the 24 hour follow-up diffusion weighted imaging (DWI, final infarct core) and correlations with clinical outcome (modified Rankin Scale (mRS) at 90 days) with the Spearman method. The random model outperformed the inter-expert agreement ((1) to (2)) and the majority model ((1) to (3)) (dice 0.51±0.04 vs 0.36±0.05 (P<0.0001) vs 0.45±0.05 (P<0.0001)). The random model predicted volume correlated with clinical outcome (0.19, P<0.05), whereas the median expert volume and majority model volume did not. There was no significant difference when comparing the volume correlations between random model, median expert volume, and majority model to 24 hour follow-up DWI volume (P>0.05, n=51). The random model for ischemic injury delineation on non-contrast CT surpassed the inter-expert agreement ((1) to (2)) and the performance of the majority model ((1) to (3)). We showed that the random model volumetric measures of the model were consistent with 24 hour follow-up DWI.

Scoring system for unstable spinal gunshot wounds: the study protocol

To present a research protocol for development of new scoring system for unstable spinal gunshot wounds based on inter-expert agreement assessment. To create a new tactical classification, we will distinguish and analyze clinical and CT data of patients with thoracolumbar spinal gunshot wounds. The Delphi method will be used to collaborate between several surgeons. A three-stage study will result a questionnaire (for 30 clinical cases). We will develop tactical scoring system and analyze statistical data (kappa). Various classifications have been developed for closed spinal injuries. These systems describe the nature of injury and allow one to develop tactical decisions for further actions. Another mechanism of injuries following gunshot wounds does not allow the classification of closed injuries to be adequately applied in some cases. Indeed, spinal structures follow either direct passage of a wounding projectile through the spine or transferring the energy of this projectile in contrast to classical compression, distraction and rotational-translation mechanisms typical for closed trauma.

Non-inferiority of deep learning ischemic stroke segmentation on non-contrast CT within 16-hours compared to expert neuroradiologists

We determined if a convolutional neural network (CNN) deep learning model can accurately segment acute ischemic changes on non-contrast CT compared to neuroradiologists. Non-contrast CT (NCCT) examinations from 232 acute ischemic stroke patients who were enrolled in the DEFUSE 3 trial were included in this study. Three experienced neuroradiologists independently segmented hypodensity that reflected the ischemic core on each scan. The neuroradiologist with the most experience (expert A) served as the ground truth for deep learning model training. Two additional neuroradiologists’ (experts B and C) segmentations were used for data testing. The 232 studies were randomly split into training and test sets. The training set was further randomly divided into 5 folds with training and validation sets. A 3-dimensional CNN architecture was trained and optimized to predict the segmentations of expert A from NCCT. The performance of the model was assessed using a set of volume, overlap, and distance metrics using non-inferiority thresholds of 20%, 3 ml, and 3 mm, respectively. The optimized model trained on expert A was compared to test experts B and C. We used a one-sided Wilcoxon signed-rank test to test for the non-inferiority of the model-expert compared to the inter-expert agreement. The final model performance for the ischemic core segmentation task reached a performance of 0.46 ± 0.09 Surface Dice at Tolerance 5mm and 0.47 ± 0.13 Dice when trained on expert A. Compared to the two test neuroradiologists the model-expert agreement was non-inferior to the inter-expert agreement, p < 0.05. The before, CNN accurately delineates the hypodense ischemic core on NCCT in acute ischemic stroke patients with an accuracy comparable to neuroradiologists.

Interexpert agreement between neuroradiologists in the diagnosis of middle cerebral artery stroke by computed tomography

Background. To evaluate the diagnostic accuracy of radiologists with a wide range of experience and training in emergency neuroradiology and other specialty hospitals, as well as the level of inter-experts’ agreement regarding CT signs of ischemic stroke.Material and methods. Our study included a retrospective cohort of 100 patients who underwent NCCT, CT angiography (mCTA) and CT perfusion in St. Petersburg clinical hospital. Fifty patients had confirmed middle cerebral artery ischemic stroke. The study also randomly included fifty patients in whom this diagnosis was excluded, including on the basis of dynamic follow-up as part of further clinical evaluation and follow-up CT scan of the brain. All 100 CT were reviewed by 21 radiologists who independently assessed each study for CT evidence of ischemic stroke. Statistical analysis included the Kappa-Fleiss coefficient ( ) calculation of agreement for each sign of ischemic stroke, specificity, sensitivity, and accuracy of detection of these changes.Results. The staff radiologist’s mean experience in emergency neuroradiology affects both diagnostic performance and consistency for detection of most CT features of ischemic stroke. At the same time, experience does not affect the diagnostic efficiency of radiologists with no experience in stroke assessment. The most highly specific and reproducible sign of ischemic stroke was the hyperdense middle cerebral artery sign, and the most ambiguous diagnostic sign was the gyral effacement. At the same time, all radiologists, regardless of length of service record and experience in emergency neuroradiology, are prone to over-diagnosis.Conclusion. Further search for tools and solutions is needed to level out interexpert variability regarding assessment of ischemic CT signs, since this problem has high clinical importance.

Abstract TMP69: Random Rater Sampling For Deep Learning Algorithm To Segment Acute Ischemic Stroke On Non-contrast Computed Tomography

Introduction: The delineation of volume and location of acute ischemic brain tissue (AIBT) on Non-Contrast CT (NCCT) increases the efficiency of endovascular treatment decisions. However, the manual segmentation of the AIBT on NCCT is a challenging task and suffers from low inter-expert agreement. Whether supervised deep convolutional neural networks (CNN) are more accurate than expert raters remain to be determined, and the optimal ground truth segmentation of AIBT is unclear given the low inter-expert agreement. We hypothesized that randomly sampling ground truth segmentations of expert raters would enable the CNN to better approximate an accurate ground truth and increase performance. Methods: The data set consisted of 200 NCCT images (Figure 1a)) of acute ischemic stroke patients presenting within 6-16h and consenting to the DEFUSE3 trial. Three experienced neuroradiologists manually segmented the AIBT (Figure 1b)1.-3.). The aggregated validation sets of 5-fold-cross-validation were used to compare the (i) average inter-expert agreement, the average performance of (ii) three individual CNNs trained on each rater and (iii) one CNN trained with random rater sampling. Results: (iii) Random rater sampling (Figure 1c)) lead to a CNN performance superior to (ii) the average performance of individually trained CNNs. The reliability of a CNN goes beyond the human inter-expert agreement (i) (Surface Dice at Tolerance 5mm: 0.88 versus 0.68 and 0.67, respectively; Table 1). Conclusions: The volume and location of AIBT on NCCT can reliably be segmented by a CNN. The agreement between a randomly chosen rater and the CNN predictions surpasses the inter-expert agreement (Table 1).

Radiofrequency Ablation in Chronic Pain Syndromes

Chronic pain is a frequent, intricate, and adverse condition that has a considerable influence on individuals and society at large. In India, its prevalence is around 20%. Although a spectrum of conservative treatment modalities is available, a significant proportion of patients with chronic pain syndromes remain refractory and require surgical intervention. In these groups of patients, radiofrequency ablation (RFA) techniques are safe minimally invasive treatments and provide significant and durable pain relief. Thus, we aimed to formulate the Indian Society for the Study of Pain (ISSP) guidelines for the management of chronic pain syndromes with various RFA techniques. An in-depth literature review by experts in Pain Medicine practising in India, was used to produce 16 statements across 4 common chronic pain syndromes, including knee pain, headache and facial pain, lumbar facet joint pain, and sacroiliac joint pain. The quality of evidence was assessed with the Third US Preventive Service Task Force guidance document and the strength of the recommendation was determined by the Delphi consensus process. The level of evidence for most of the statements was I. Moreover, for most statements, the level of agreement between the experts was good (≥80% of the experts). The ISSP guidelines for the management of chronic pain syndromes are developed by experts in pain medicine. For most of the statements, the highest level of evidence was available and inter-expert agreement was good. However, further high-quality research is required to formulate more inclusive guidelines in this evolving pain medicine speciality.

Automated Inline Myocardial Segmentation of Joint T1 and T2 Mapping Using Deep Learning.

To develop an artificial intelligence (AI) solution for automated segmentation and analysis of joint cardiac MRI short-axis T1 and T2 mapping. In this retrospective study, a joint T1 and T2 mapping sequence was used to acquire 4240 maps from 807 patients across two hospitals between March and November 2020. Five hundred nine maps from 94 consecutive patients were assigned to a holdout testing set. A convolutional neural network was trained to segment the endocardial and epicardial contours with use of an edge probability estimation approach. Training labels were segmented by an expert cardiologist. Predicted contours were processed to yield mapping values for each of the 16 American Heart Association segments. Network segmentation performance and segment-wise measurements on the testing set were compared with those of two experts on the holdout testing set. The AI model was fully integrated using open-source software to run on MRI scanners. A total of 3899 maps (92%) were deemed artifact-free and suitable for human segmentation. AI segmentation closely matched that of each expert (mean Dice coefficient, 0.82 ± 0.07 [SD] vs expert 1 and 0.86 ± 0.06 vs expert 2) and compared favorably with interexpert agreement (Dice coefficient, 0.84 ± 0.06 for expert 1 vs expert 2). AI-derived segment-wise values for native T1, postcontrast T1, and T2 mapping correlated with expert-derived values (R 2 = 0.96, 0.98, and 0.87, respectively, vs expert 1, and 0.97, 0.99, and 0.92 vs expert 2) and fell within the range of interexpert reproducibility (R 2 = 0.97, 0.99, and 0.90, respectively). The AI model has since been deployed at two hospitals, enabling automated inline analysis. Automated inline analysis of joint T1 and T2 mapping allows accurate segment-wise tissue characterization, with performance equivalent to that of human experts.Keywords: MRI, Neural Networks, Cardiac, Heart Supplemental material is available for this article. © RSNA, 2022.

Inter-expert agreement on indications for antibiotic therapy in older adults admitted to French hospital through an emergency department

BackgroundAround one third of older adults with infections have an atypical presentation upon admission to an emergency department (ED). ObjectiveTo evaluate the level of agreement between experts from several disciplines on the indication for antibiotic therapy for a bacterial infection in older patients presenting at an ED, and to describe the characteristics of the infections. MethodsBased on comprehensive medical records, three experts (a geriatrician, an emergency physician (EP), and an infectious disease specialist (IDS)) determined independently and then jointly whether a patient presenting at the ED had a bacterial infection requiring antibiotic therapy. Inter-expert agreement was expressed as a fixed-marginal Fleiss’ kappa (κ). ResultsOf the 444 medical records included, the consensus meeting found that 114 (25.7%) had an indication for antibiotics, 327 (73.6%) did not have an indication, and 3 could not be classified. The overall level of agreement was 85.2%, and κ[95%CI] was 0.64 [0.57–0.72] (p < 0.001). The level of agreement between the geriatrician and the IDS (89.41%, κ0.73, 95%CI [0.62–0.85] (p < 0.001)) was higher than that between the geriatrician and the EP (83.56%, κ0.62, 95%CI [0.51–0.73] (p < 0.001)) and between the IDS and the EP (82.66%, κ0.59, 95%CI [0.48–0.70] (p < 0.001)). The levels of agreement between the final adjudication, was higher for the geriatrician, and IDS respectively 94.1% (κ0.85, 95%CI [0.74–0.97] (p < 0.001) and 94.4% (κ0.86, 95%CI [0.74–0.97] (p < 0.001)). 114 (25.7%) patients had a bacterial infection (mostly lung infections (n = 55, 48.2%) and urinary tract infections (n = 25, 21.9%)), and 28 patients (6.3%) had a viral infection. ConclusionOur results highlighted substantial agreement between members of a multidisciplinary expert panel.

Interexpert Agreement Between Emergency Neuroradiologists with Different Levels of Experience in the Rating of ASPECTS

Fifteen radiologists independently evaluated fifty non-contrast brain computed tomography of patients with middle cerebral artery (MCA) stroke. Within-group agreement was either negligible or moderate, and the best inter-expert agreement was observed between neuroradiologists with 1 to 3 years of experience. Our study allows us to conclude that ASPECTS is not an ideal tool for standardizing the diagnosis of MCA stroke, therefore, it is necessary to develop new models of standardization systems, as well as the use of hydride diagnostics, including automatic analysis.

АДАПТАЦИЯ И ВАЛИДАЦИЯ РУССКОЯЗЫЧНОЙ ВЕРСИИ ОПРОСНИКА STRONGKIDS ДЛЯ ОЦЕНКИ НУТРИТИВНОГО РИСКА У ДЕТЕЙ

Malnutrition (MN) is common in children with acute and chronic illnesses. The use of questionnaires to assess nutritional risk in children admitted to hospital is a simple and quick way to identify patients who need nutritional advice and nutritional support. Various English-language screening questionnaires, such as STRONGkids nutritional risk screening tool for children, in particular, have been developed and validated. The purpose of the study is to adapt the STRONGkids nutritional risk screening tool for the Russian language speaking audience and to determine the reliability and validity of its new language version. Materials and methods of research: linguistic ratification of the STRONGkids questionnaire was carried out and the Russian language version was created. This single-center prospective study included 419 children admitted from October, 2019 to March, 2021 for treatment in Therapeutic Pediatric (cardiac, nephrological, gastroenterological, pulmonological etc.) and Surgical Pediatric (abdominal, thoracic, otorhinolaryngological, cardiosurgical etc.) Departments of the National Medical Research Center for Children’s Health (Moscow, Russia) with a hospital stay longer than a single day. The reliability of the questionnaire was assessed in 50 children. The anthropometric indicators were used to assess the current validity; the duration of hospitalization was used to assess the predictive validity. Body mass index-for-age (BMI-for-age) &lt;-2SD and Length/height-for-age &lt;-2SD z-scores were criteria for acute and chronic MN, respectively. Results: almost complete (κ=0.84) inter-expert agreement was found during the evaluation of reliability of the STRONGkids questionnaire. The validation study included 419 patients (51% boys/49% girls) aged from 1 month till 17 years and 11 months old (8 years old as median) with hospitalization duration from 1 to 100 days (10 days as median), of which 314 (75%) and 105 (25%) therapeutic pediatric and surgical pediatric patients, respectively. The frequency of acute and chronic MN at admission was 9.1% and 7.4%, respectively, and the percentage of children at low, moderate and high risk (assessed by the questionnaire) was 22.7%, 66.6% and 10.7%, respectively. Children at nutritional risk had statistically significantly (p&lt;0.05) lower BMI-for-age and Length/height-for-age z-scores, greater incidence of MN and longer hospital stays than patients without nutritional risk. A chronic disease addendum, complications and high nutritional risk according to the questionnaire are independent unfavorable factors for an increase in the duration of hospitalization. Conclusion: the Russian language version of the STRONGkids nutritional risk screening tool for children was adapted and validated to assess nutritional risk in children admitted to hospital for treatment. Since right now it can be of use for identifying patients at high risk of MN for the purpose of their additional examination and timely administration of nutritional support and can be used both in routine pediatric clinical practice and in purpose of further scientific research.

Экспертная валидация стандартизированного описания моделей организации службы лабораторной диагностики

Introduction. The growing demand for the availability and quality of clinical laboratory tests has necessitated the search for new approaches to organizing and management of laboratory services. At the same time, needs a systematic description of the laboratory services organization models as a background for further improvement. Objective: to validate the description of the laboratory services organization models by standardized characteristics. Methods. A sociological study carried out on the principles of a systematic approach. Research methods: sociological (expert assessments), analytical, statistical. Results and discussion. The inter-expert agreement for the centralization (consolidation) model was 0,626 (95% CI 0,516–0,711), outsourcing (decentralization) – 0,747 (95% CI 0,672–0,804), horizontal integration – 0,784, 95% CI 0,720–0,833, mobile (pointto-care) model – 0,883 (95% CI 0,848–0,909). The first value interpreted as doubtful, significant adjustments made to the model characteristics. The second and third values – as sufficient, minimal adjustments made. Fourth – as good, the description was not changed. The final SWOT-analysis matrices for four models formed. Conclusions. Thanks to expert validation, a methodical change from an empirical to a scientifically based standardized description of the four basic laboratory services organization models made.

Comparison of Intra- and Inter-Observer Consistency (Intra-Expert Reliability, Inter-Expert Reliability) in Assessing the Extent of COVID-19 Pneumonia Lesions on Chest Computed Tomography

Background. The SARS-CoV-2 pandemic has set new challenges for the radiological community: early diagnosis of interstitial pneumonia, estimation of its severity, and dynamics. Computed tomography has become the method of choice for assessing lung tissue in COVID-19 patients, which is due to the low sensitivity of radiography in detecting a decrease in airiness by the ground glass opacity type. The criteria for assessing visual signs of lung tissue damage often have a degree of subjectivity, and the conclusion based on them affects the patients management tactics. Aims to determine the inter- and intra-expert consistency in the assessment of the percentage of lesions and the CT stage of COVID-19 pneumonia among experts-radiologists with different experience, to analyze the level of precision depending on the prevalence of the lesion and other factors. Materials and methods. The research analyzed CT scans of 221 patients with confirmed SARS-CoV-2 by PCR. Patients with additional lung pathology and some patients with lesions up to 50% were excluded to create a uniform degree sample of 100 patients. Four expert radiologists determined the percentage of lung damage and the CT stages. The results of the expert assessment are analyzed using the methods of classical descriptive statistics and the analysis of intra-and inter-expert consistency. Results. The correlation of the level of lung damage, when evaluating intra-expert convergence (after 6 months), as a percentage between the first and second reading was R = 0.86 (p 0.05) for expert 1 (high level of training), R = 0.84 (p 0.05) for expert 2 (high level of training). Within the expert agreement, Kappa (for K-degree) was 0.54 for expert 1 and 0.46 for expert 2, which corresponds to a moderate level of consistency. When assessing inter-expert convergence, the connectivity between the level of lung damage as a percentage between experts 1 and 2 (high level of training) was R = 0.87 (p 0.05), between experts 3 and 4 (low level of training) R = 0.78 (p 0.05). The measure of inter-expert agreement Kappa was 0.51 for experts 1 and 2 and 0.56 for experts 3 and 4. The average assessment of experts in the sample varied up to 4.5%, and when analyzing the differences in expert opinions, the difference varied evenly, both in the degree of increase and decrease in the volume of the lesion. Conclusions. In spite of the high level of correlation in the assessment of the percentage of lesion, the convergence of the Cap on the CT degree was moderate, not significantly differing from the degree of expert training. More often, differences in the level of damage are observed in non-classical patterns: reverse halo, curvilinear lines, etc. The difference in opinions doesnt represent a systematic error. Hereby the expert assessment of the volume of lung damage empirically has a moderate, insufficient level of reliability.

Survey among Russian doctors on routine electroencephalography

The aim of the work was to assess how accurately specialists, performing EEG, apply the recommendations in their daily practice.Materials and Methods. An anonymous survey was conducted of 210 specialists in functional diagnostics and neurologists performing routine electroencephalography. The survey consisted of nine mandatory single-choice questions.Results. Only 9 respondents out of 210 (4.3%) perform routine EEG completely in accordance with the Recommendations of the Expert Council of Neurophysiology of the Russian Anti-Epileptic League on performing the routine EEG.' Of these, only three specialists (1.4% of the total number of respondents) answered that a functional diagnostics doctor is present throughout the entire examination.Conclusion: Despite the wide distribution and extensive experience in the use of EEG, there is no uniform practice of its implementation by doctors in the Russian Federation. Standardization of the methodology of its performance will increase confidence in EEG results among doctors of clinical specialties, increase the degree of inter-expert agreement among functional diagnostics physicians, and improve the quality of diagnosis of central nervous system diseases. To do this, it is necessary to develop webinar and educational programs on the methodology of EEG under the auspices of professional associations of doctors of functional diagnostics, neurophysiologists, neurologists.

Comparison of thoracic ultrasonography and thoracic radiography to detect active infectious bronchopneumonia in hospitalized dairy calves.

BackgroundThe best test between thoracic ultrasonography (TUS) and thoracic radiography (TR) or the best combination of tests (series or parallel) to detect active infectious bronchopneumonia (BP) in hospitalized dairy calves remains unknown.Hypothesis/ObjectivesTo estimate performances of TUS and TR to detect active BP in hospitalized dairy calves and to determine the best strategy for using these tests based on a panel diagnosis method (PDM). Performances of TUS and TR were hypothesized to be equivalent.AnimalsFifty hospitalized dairy calves (≥7 days old; ≤100 kg; standing; pCO2 ≥ 53 mm Hg; any reason of presentation).MethodsEach calf prospectively and sequentially underwent physical examination, thoracic auscultation, blood analyses, and TUS and TR. Three blinded experts determined whether active BP was present/absent based on PDM. Krippendorff's alpha measured interexpert agreement. The sensitivities (Se) and specificities (Sp) of TUS and TR alone and in series or parallel were compared (McNemar's test; P < .05).ResultsInterexpert agreement was moderate at 0.58 (95%CI: 0.42; 0.73). The Se and Sp of TUS were 0.84 (95%CI: 0.60; 0.97) and 0.74 (95%CI: 0.57; 0.86), respectively. The Se and Sp of TR were 0.89 (95%CI: 0.67; 0.99) and 0.58 (95%CI: 0.39; 0.75), respectively. No significant difference was found in the Se and Sp of TUS and TR when analyzed alone, in series or in parallel.ConclusionThoracic ultrasonography or TR alone equally detected active BP in hospitalized dairy calves. Series or parallel analysis provided no additional benefit. Its ease of use and widespread accessibility support using TUS as a first‐line test to detect active BP in hospitalized dairy calves.

Fully automatic detection and classification of phytoplankton specimens in digital microscopy images

Background and objectiveThe proliferation of toxin-producing phytoplankton species can compromise the quality of the water sources. This contamination is difficult to detect, and consequently to be neutralised, since normal water purification techniques are ineffective. Currently, the water analyses about phytoplankton are commonly performed by the specialists with manual routine analyses, which represents a major limitation. The adequate identification and classification of phytoplankton specimens requires intensive training and expertise. Additionally, the performed analysis involves a lengthy process that exhibits serious problems of reliability and repeatability as inter-expert agreement is not always reached. Considering all those factors, the automatization of these analyses is, therefore, highly desirable to reduce the workload of the specialists and facilitate the process. MethodsThis manuscript proposes a novel fully automatic methodology to perform phytoplankton analyses in digital microscopy images of water samples taken with a regular light microscope. In particular, we propose a method capable of analysing multi-specimen images acquired using a simplified systematic protocol. In contrast with prior approaches, this enables its use without the necessity of an expert taxonomist operating the microscope. The system is able to detect and segment the different existing phytoplankton specimens, with high variability in terms of visual appearances, and to merge them into colonies and sparse specimens when necessary. Moreover, the system is capable of differentiating them from other similar objects like zooplankton, detritus or mineral particles, among others, and then classify the specimens into defined target species of interest using a machine learning-based approach. ResultsThe proposed system provided satisfactory and accurate results in every step. The detection step provided a FNR of 0.4%. Phytoplankton detection, that is, differentiating true phytoplankton from similar objects (zooplankton, minerals, etc.), provided a result of 84.07% of precision at 90% of recall. The target species classification, reported an overall accuracy of 87.50%. The recall levels for each species are, 81.82% for W. naegeliana, 57.15% for A. spiroides, 85.71% for D. sociale and 95% for the ”Other” group, a set of relevant toxic and interesting species widely spread over the samples. ConclusionsThe proposed methodology provided accurate results in all the designed steps given the complexity of the problem, particularly in terms of specimen identification, phytoplankton differentiation as well as the classification of the defined target species. Therefore, this fully automatic system represents a robust and consistent tool to aid the specialists in the analysis of the quality of the water sources and potability.

Interobserver agreement of the anatomic and physiological classification system for adult congenital heart disease

Consensus definitions for AP categories were developed with input from 4 experts. Research assistants (RAs) assigned AnatC/PhyS for patients in the Boston ACHD Biobank, a prospectively enrolled cohort of ambulatory ACHD patients ≥18 years old seen between 2012 and 2019. Two (of 4) expert reviewers then independently assigned AnatC/PhyS for 41 patients. Interrater reliability was assessed with linearly weighted kappa (κω) for agreement between (1) experts and (2) an RA and an expert. Experts examined disagreements and identified sources of variability and areas requiring clarification. Interexpert agreement for AnatC was excellent, with agreement on 38/41 (92.7%) cases and κω 0.88 [0.75, 1.01]. Agreement for PhyS was less robust, with consensus on 24/41 cases (59.5%), κω 0.57 [0.39, 0.75]. Expert-RA agreement was lower for AnatC (κω 0.77 [0.60, 0.95]), whereas PhyS was similar to interexpert agreement (κω 0.53 [0.34, 0.72]). There was ambiguity in the definitions of (1) arrhythmia status, (2) cyanotic CHD, and (3) valve disease. Although AnatC can be assessed reliably, that is not true for the PhyS part of the AP classification proposed in the 2018 American College of Cardiology/American Heart Association guidelines. Reliability of PhyS would be strengthened by more precise definitions readily interpretable in clinical practice.