Observational Analysis of Mistakes in Chess Initiation, Using Decision Trees

What is the inter-observer and intra-observer agreement between embryologists when selecting a single Day 5 embryo for transfer? The inter-observer and intra-observer agreement between embryologists when selecting a single Day 5 embryo for transfer was generally good, although not optimal, even among experienced embryologists. Previous research on the morphological assessment of early stage (two pronuclei to Day 3) embryos has shown varying levels of inter-observer and intra-observer agreement. However, single blastocyst transfer is now becoming increasingly popular and there are no published data that assess inter-observer and intra-observer agreement when selecting a single embryo for Day 5 transfer. This was a prospective study involving 10 embryologists working at five different IVF clinics within a single organization between July 2013 and November 2015. The top 10 embryologists were selected based on their yearly Quality Assurance Program scores for blastocyst grading and were asked to morphologically grade all Day 5 embryos and choose a single embryo for transfer in a survey of 100 cases using 2D images. A total of 1000 decisions were therefore assessed. For each case, Day 5 images were shown, followed by a Day 3 and Day 5 image of the same embryo. Subgroup analyses were also performed based on the following characteristics of embryologists: the level of clinical embryology experience in the laboratory; amount of research experience; number of days per week spent grading embryos. The agreement between these embryologists and the one that scored the embryos on the actual day of transfer was also evaluated. Inter-observer and intra-observer variability was assessed using the kappa coefficient to evaluate the extent of agreement. This study showed that all 10 embryologists agreed on the embryo chosen for transfer in 50 out of 100 cases. In 93 out of 100 cases, at least 6 out of the 10 embryologists agreed. The inter-observer and intra-observer agreement among embryologists when selecting a single Day 5 embryo for transfer was generally good as assessed by the kappa scores (kappa = 0.734, 95% CI: 0.665-0.791 and 0.759, 95% CI: 0.622-0.833, respectively). The subgroup analyses did not substantially alter the inter-observer and intra-observer agreement among embryologists. The agreement when Day 3 images were included alongside Day 5 images of the same embryos resulted in a change of mind at least three times by each embryologist (on average for <10% of cases) and resulted in a small decrease in inter-observer and intra-observer agreement between embryologists (kappa = 0.676, 95% CI: 0.617-0.724 and 0.752, 95% CI: 0.656-808, respectively).The assessment of the inter-observer agreement with regard to morphological grading of Day 5 embryos showed only a fair-to-moderate agreement, which was observed across all subgroup analyses. The highest overall kappa coefficient was seen for the grading of the developmental stage of an embryo (0.513; 95% CI: 0.492-0.538). The findings were similar when the individual embryologists were compared with the embryologist who made the morphological assessments of the available embryos on the actual day of transfer. All embryologists had already completed their training and were working under one organization with similar policies between the five clinics. Therefore, the inter-observer agreement might not be as high between embryologists working in clinics with different policies or with different levels of training. The generally good, although not optimal uniformity between participating embryologists when selecting a Day 5 embryo for transfer, as well as, the surprisingly low agreement when morphologically grading Day 5 embryos could be improved, potentially resulting in increased pregnancy rates. Future studies need to be directed toward technologies that can help achieve this. None declared. Not applicable.

Decision-making in the treatment of diaphyseal clavicle fractures: is there agreement among surgeons? Results of a survey on surgeons' treatment preferences

To evaluate the visibility of cesarean section (CS) scars by transvaginal sonography (TVS) in pregnant women, to apply a standardized approach for measuring CS scars and to test its reproducibility throughout the course of pregnancy. In this observational cohort study, 320 consecutive pregnant women with a previous cesarean delivery were examined to assess scar visibility by two independent examiners. TVS was carried out at 11-13, 19-21 and 34-36 weeks' gestation. A scar was defined as visible when an area of hypoechogenicity representing myometrial discontinuity at the anterior wall of the lower uterine segment was identified. In a subset of patients (n = 111), visible scars were measured by two independent examiners in three dimensions: scar width, depth and length as well as the residual myometrial thickness (RMT). Descriptive analysis was used to assess scar visibility, and the intraclass correlation coefficient (ICC) was calculated to show the strength of absolute agreement between two examiners for scar measurements. For RMT, a cut-off of 2.4 mm was used and measurement agreement was assessed using Cohen's kappa coefficient. The scar was visible in 284/320 cases (88.8%). Visible scars were significantly associated with anteverted uteri (P < 0.0001). Both examiners had 100% agreement on scar visibility at 12 and 20 weeks' gestation, while agreement was 96% at 34 weeks. The intra- and interobserver agreements for scar measurements were generally good (ICC 0.86 and 0.89, respectively). The kappa coefficient for the RMT was 0.27 in the first trimester, compared with 0.51 and 0.72 in the second and third trimesters, respectively. CS scars remain visible in the majority of women throughout pregnancy. They can be reproducibly measured in three dimensions when assessed by TVS in all trimesters of pregnancy. The agreement between two observers for CS scar measurement can be considered good for the first trimester, compared with relatively moderate agreement for the second and third trimesters.

The purpose of this study was to determine whether an ecological imagery intervention program would affect perception (i.e., exploratory activity and prospective control of future actions) in three elite soccer players. The imagery was adjusted to the unique action opportunities typically experienced by each player in games. A single case, multiple baseline across participants design was implemented and close-up video analyses were conducted from a series of league games. Post-intervention questionnaires and interviews were also carried out to support the video analyses. Two of the participants appeared to increase their visual exploratory activity, but only one of the participants marginally improved his performance with the ball. It was concluded that elite players can improve components of perception through ecological imagery training, but it is questionable to what extent this leads to improved prospective control of actions. It is recommended that future imagery and/or perceptual training research addresses specific types of actions more directly.

Introduction Thoracolumbar fracture classification systems have failed to demonstrate adequate comprehensiveness, or have demonstrated low reliability and reproducibility. The new AO Spine Thoracolumbar Spine Injury Classification System was recently published. It showed substantial reliability and reproducibility among the surgeons who developed it; however, an independent evaluation has not been performed. We performed an independent interobserver and intraobserver agreement evaluation of the new AO Spine Thoracolumbar Spine Injury Classification System. Patients and Methods Anteroposterior and lateral radiographs, and computed tomography scans of 70 patients with acute traumatic thoracolumbar injuries were selected and classified by six evaluators (three spine surgeons and three orthopedic surgery residents). The evaluators were unaware of the patients' identification, the original classification used in their clinical care, and the treatment they received. They classified the lesions according to the morphologic grading of the new AO Spine thoracolumbar spine injury classification system as an A, B, or C type injury; in cases with type A or type B injuries, an analysis for A or B subtypes lesions was also performed. After a 6-week interval, the 70 cases were presented in a random sequence to the same evaluators for repeat evaluation. The Kappa coefficient (κ) was used to determine the interobserver and intraobserver agreement. Results All the different fracture patterns as described by the new AO Spine thoracolumbar spine injury classification system were identified by at least one evaluator. The interobserver reliability was substantial when considering the fracture types (A, B, or C), with k = 0.62 (0.57–0.66). The interobserver agreement when considering the subtypes was moderate; k = 0.55 (0.52–0.57). The intraobserver reproducibility was also substantial, with 85.95% full intraobserver reproducibility considering the fracture type, with k = 0.77 (0.72–0.83), and it was also substantial when considering subtypes with 75.71% full agreement and k = 0.71 (0.67–0.76). No significant differences were observed between spine surgeons and orthopedic residents in the overall interobserver reliability and intraobserver reproducibility, or in the interobserver and intraobserver agreement of specific A, B, or C types of injuries. Conclusion Our study showed substantial agreement among different observers and by the same observer on separate occasions. Considering that this classification allows a better communication among physicians than previous schemes, it could be employed as a practical tool for everyday clinical use. Furthermore, clinical studies employing this classification should be considered more accurate, and their conclusions become more relevant to guide treatment decision making than studies using other classification schemes, especially in case of multicenter studies.

INTRODUCTION: Since its introduction, the RENAL nephrometry score has proven to be an effective tool in medical decision-making and surgical planning for renal tumors and has been shown to be predictive of several important oncologic and operative outcomes. However, some studies have reported high interobserver variability, particularly within the endophycity or “E” portion of RENAL score. This variability can undermine the utility of the RENAL score in clinical management, as it may lead to inconsistent risk stratification and treatment recommendations. Here, we propose the use of center-based endophycity (CBE) scoring in place of the “E” score within RENAL as a means of facilitating interobserver agreement while maintaining utility. METHODS: Medical records for patients who underwent radical or partial nephrectomy for suspected renal malignancy at a single institution over a 9-year span were retrospectively collected. Exclusion criteria included unavailable imaging, multifocal disease, and incomplete pathology data. Preoperative imaging was evaluated with six independent observers manually assigning RENAL scores to each case, and three observers assigning CBE scores. CBE scores were determined by approximating the center of the tumor and assessing whether it was interior or exterior to the corticomedullary boundary of the kidney. Exophytic and endophytic tumors were given a score of 1 and 2 respectively. These scores were then integrated in place of the “E” score of RENAL for analysis. Kappa coefficient was used to analyze interobserver agreement, and logistic regression was used for predicting outcomes. RESULTS: A total of 288 cases met the inclusion criteria for this study. CBE scoring (kappa coefficient 0.77) had significantly higher interobserver agreement than RENAL “E” (kappa coefficient 0.17, p &amp;lt; 0.0001). Both traditional RENAL and RENAL with CBE were predictive of ISUP grade (p &amp;lt; 0.0001, p &amp;lt; 0.0001), tumor necrosis (p &amp;lt; 0.0001, p &amp;lt; 0.0001), and perioperative blood loss (p = 0.001, p &amp;lt; 0.0001). Traditional RENAL was predictive of malignancy (p = 0.001), while RENAL with CBE was not (p = 0.058). However, CBE independently predicted malignancy (p = 0.039), whereas RENAL “E” did not (p = 0.365). CONCLUSIONS: These results suggest that the RENAL “E” score is prone to particularly low interobserver agreement. When integrated in place of RENAL “E”, CBE scoring represents a promising modification to the RENAL nephrometry score, offering a more reproducible method for assessing endophycity in renal tumors while preserving much of the predictive power of the original model. These findings have important implications for improving the accuracy and consistency of preoperative renal tumor assessments. Further studies are needed to elucidate currently insignificant yet promising predictive data points from this study and to confirm our findings in other cohorts. Citation Format: Keenan Moore, Nicholas Heller, Clara Goebel, Jayant Siva, Jamee Schoephoerster, Griffin Struyk, Nitin Venkatesh, Beatriz Lopez Morato, Christopher Weight. Enhancing interobserver agreement in RENAL nephrometry scoring through the integration of center-based endophycity measurements [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5969.

Agreement study. To perform an independent interobserver and intraobserver agreement evaluation of the new AOSpine Thoracolumbar Spine Injury Classification System. The new AOSpine Thoracolumbar Spine Injury Classification System was recently published. It showed substantial reliability and reproducibility among the surgeons who developed it; however, an independent evaluation has not been performed. Anteroposterior and lateral radiographs, and computed tomographic scans of 70 patients with acute traumatic thoracolumbar injuries were selected and classified using the morphological grading of the new AOSpine Thoracolumbar Spine Injury Classification System by 6 evaluators (3 spine surgeons and 3 orthopedic surgery residents). After a 6-week interval, the 70 cases were presented in a random sequence to the same evaluators for repeat evaluation. The Kappa coefficient (κ) was used to determine the interobserver and intraobserver agreement. The interobserver reliability was substantial when considering the fracture type (A, B, or C), with a κ= 0.62 (0.57-0.66). The interobserver agreement when considering the subtypes was moderate; κ= 0.55 (0.52-0.57). The intraobserver reproducibility was also substantial, with 85.95% full intraobserver reproducibility considering the fracture type, with κ= 0.77 (0.72-0.83), and was also substantial when considering subtypes with 75.71% full agreement and κ= 0.71 (0.67-0.76). No significant differences were observed between spine surgeons and orthopedic residents in the overall interobserver reliability and intraobserver reproducibility, or in the inter- and intraobserver agreement of specific A, B, or C types of injuries. This classification allows adequate agreement among different observers and by the same observer on separate occasions. Future prospective studies should evaluate whether this classification improves clinical decision making.

BackgroundMyopic maculopathy (MM) is one of the major causes of visual impairment and irreversible blindness in eyes with pathologic myopia (PM). However, the classification of each type of lesion associated with MM has not been determined. Recently, a new MM classification system, known as the ATN grading and classification system, was proposed; it is based on the fundus photographs and optical coherence tomography (OCT) images and includes three variable components: atrophy (A), traction (T), and neovascularization (N). This study aimed to perform an independent evaluation of interobserver and intraobserver agreement for the recently developed ATN grading system for MM.MethodsThis was a retrospective study. Fundus photographs and OCT images of 125 patients (226 eyes) with various MMs were evaluated and classified using the ATN grading of the new MM classification system by four blinded and independent evaluators (2 attending ophthalmologists and 2 ophthalmic residents). All cases were randomly re-evaluated by the same observers after an interval of 6 weeks. The kappa coefficient (κ) and 95% confidence interval (CI) were used to determine the interobserver and intraobserver agreement.ResultsThe interobserver agreement was substantial when considering the maculopathy type (A, T, and N). The weighted Fleiss κ values for each MM type (A, T, and N) were 0.651 (95% CI: 0.602–0.700), 0.734 (95% CI: 0.689–0.779), and 0.702 (95% CI: 0.649–0.755), respectively. The interobserver agreement when considering the subtypes was good or excellent, except for stages A1, A2, and N1, in which the weighted κ value was less than 0.6, with moderate agreement. The intraobserver agreement of types and subtypes was excellent, with κ > 0.8. No significant differences were observed between the attending ophthalmologists and residents for interobserver reliability or intraobserver reproducibility.ConclusionsThe ATN classification allows an adequate agreement among ophthalmologists with different qualifications and by the same observer on separate occasions. Future prospective studies should further evaluate whether this classification can be better implemented in clinical decision-making and disease progression assessments.

Inter-observer agreement between dermatologists and oncologists in assessing dermatological toxicities in patients with metastatic colorectal cancer treated by cetuximab-based chemotherapies: A pilot comparative study

Purpose: To evaluate interobserver and intertest agreement between optical coherence tomography (OCT) and retinography in the detection of glaucoma through a telemedicine program. Methods: A stratified sample of 4113 individuals was randomly selected, and those who accepted underwent examination including visual acuity, intraocular pressure (IOP), non-mydriatic retinography, and imaging using a portable OCT device. Participants’ data and images were uploaded and assessed by 16 ophthalmologists on a deferred basis. Two independent evaluations were performed for all participants. Agreement between methods was assessed using the kappa coefficient and the prevalence-adjusted bias-adjusted kappa (PABAK). We analyzed potential factors possibly influencing the level of agreement. Results: The final sample comprised 1006 participants. Of all suspected glaucoma cases (n = 201), 20.4% were identified in retinographs only, 11.9% in OCT images only, 46.3% in both, and 21.4% were diagnosed based on other data. Overall interobserver agreement outcomes were moderate to good with a kappa coefficient of 0.37 and a PABAK index of 0.58. Higher values were obtained by experienced evaluators (kappa = 0.61; PABAK = 0.82). Kappa and PABAK values between OCT and photographs were 0.52 and 0.82 for the first evaluation. Conclusion: In a telemedicine screening setting, interobserver agreement on diagnosis was moderate but improved with greater evaluator expertise.

Intra- and Interobserver Agreement during the Assessment of the Different Stages of Root Development Using 4 Radiographic Classifications

A new pilon fracture classification system based on CT scan data was recently published, showing almost perfect interobserver and intraobserver agreement among the authors who developed it. However, an independent assessment has not been done. To do an independent agreement evaluation of the new pilon fracture classification system with physicians with different levels of expertise in the management of pilon fractures. Seventy-one cases of acute pilon fracture were retrospectively collected. Fractures were classified by six evaluators (three foot and ankle surgeons and three orthopaedic surgery residents) using CT scans according to the morphological grading of the new pilon fracture classification system developed by Leonetti et al. Cases were presented to the same evaluators in a random sequence after a 6-week interval to determine intraobserver agreement. The kappa coefficient (κ) was used to determine agreement among evaluators. The interobserver agreement was substantial regarding the main fracture type (I, II, III, or IV), with an overall κ value of 0.69 (0.65 to 0.72). When including the II and III subtypes, the overall agreement was still substantial, with a κ value of 0.61 (95% confidence interval: 0.58 to 0.64). The intraobserver agreement was substantial when considering the main fracture categories (I, II, III, or IV), with a κ value of 0.78 (confidence interval: 0.72 to 0.84), and full agreement at the type level was observed in 76% (324/426) of evaluations. There was no notable difference between the foot and ankle surgeons and orthopaedic surgery residents in the interobserver and intraobserver agreement. The new classification system demonstrated substantial interobserver and intraobserver agreement between evaluators with different levels of expertise in the management of pilon fractures. Prospective studies should be done to evaluate its prognostic value and utility in clinical practice.

ObjectiveDistal clavicle fracture classification directly affects the treatment decisions. It is unclear whether the classification systems implemented differ depending on surgeons' backgrounds. This study aimed to compare the interobserver agreement of four classification systems used for lateral clavicle fractures by shoulder specialists and general trauma surgeons.MethodsRadiographs of 20 lateral clavicle fractures representing a full spectrum of adult fracture patterns were analyzed by eight experienced shoulder specialists and eight general trauma surgeons from 10 different hospitals. All cases were graded according to the Orthopedic Trauma Association (OTA), Neer, Jäger/Breitner, and Gongji classification systems. To measure observer agreement, Fleiss' kappa coefficient (κ) was applied and assessed.ResultsWhen only X‐ray films were presented, both groups achieved fair agreement. However, when the 3D‐CT scan images were provided, improved interobserver agreement was found in the specialist group when the OTA, Jäger/Breitner, and Gongji classification systems were used. In the generalist groups, improved agreement was found when using the Gongji classification system. In terms of interobserver reliability, the OTA, Neer, and Jäger/Breitner classification systems showed better agreement among shoulder specialists, while a slightly lower level of agreement was found using the Gongji classification system. For the OTA classification system, interobserver agreement had a mean kappa value of 0.418, ranging from 0.446 (specialist group) to 0.402 (generalist group). For the Neer classification system, interobserver agreement had a mean kappa value of 0.368, ranging from 0.402 (specialist group) to 0.390 (generalist group). For the Jäger/Breitner classification system, the inter‐observer agreement had a mean kappa value of 0.380, ranging from 0.413 (specialist group) to 0.404 (generalist group). For the Gongji classification system, interobserver agreement had a mean kappa value of 0.455, ranging from 0.480 (specialist group) to 0.485 (generalist group).ConclusionGenerally speaking, 3D‐CT scans provide a richer experience that can lead to better results in most classification systems of lateral clavicle fractures, highlighting the value of digitization and specialization in diagnosis and treatment. Competitive interobserver agreement was exhibited in the generalist group using the Gongji classification system, suggesting that the Gongji classification is suitable for general trauma surgeons who are not highly experienced in the shoulder field.

Stereotactic body radiation therapy (SBRT) is an effective treatment modality for inoperable early-stage lung cancer or metastatic lung lesions. Post-SBRT, acute radiological lung changes sometimes mimic tumor progression, so over-investigation may be applied. We aimed to reveal the interobserver agreement among physicians regarding acute radiographic changes on CT of the thorax obtained shortly after SBRT MATERIALS AND METHODS: Radiologic images of 20lesions treated with SBRT were evaluated for acute lung changes. Two physicians, one senior and one junior, from diagnostic radiology, radiation oncology, nuclear medicine, and chest disease departments reviewed these images. The final interpretations were categorized as stable, regression/consolidation, progressive disease, and SBRT-related changes. The evaluations of the physicians were compared with the experienced reference radiation oncologist. The gold standard was accepted as the reference physician's final score. Unweighted Cohen's kappa(κ) coefficient was used for assessing interobserver agreement between physicians. The evaluations of the physicians were compared with the reference radiation oncologist. The strongest coherence coefficient was found with the senior radiation oncologist (κ: 0.72).The kappa coefficients between the junior radiation oncologist, junior nuclear medicine physician, and the reference physician were 0.61 and 0.55, respectively. The disciplines with the lowest kappa coefficients were junior chest disease and senior radiologist, and the kappa values were 0.37 and 0.44, respectively. Disciplines dealing with lung cancer treatment may not be aware of the various radiologic changes after SBRT or inexperienced in interpreting them from recurrence. Therefore, physicians must have detailed radiotherapy information such as planning target volume (PTV), dose/fractionation, etc. In addition, final evaluations should be performed in the multidisciplinary team dealing with the treatment of the patient.

To evaluate the intraobserver and interobserver reliability and main influencing factors of the King, Lenke, and PUMC classification systems for idiopathic scoliosis (IS) using radiographs that had not been premeasured. Fifty-six patients with IS, 9 male and 47 female, aged 15.4 (11 - 18), underwent preoperative X-ray photography of spine standing full-length posteroanterior and lateral films and left and right supine side-bending radiographs. The films were read by four orthopedic surgeons independently to do measurement and typing according to the King, Lenke, and PUMC classification systems respectively. Two weeks later, the sequence of the X-ray films was re-ordered and the 4 surgeons read them and did measurement and typing once more. The average percentage of intraobserver and interobserver agreement was calculated. Kappa coefficients were used to determine the intraobserver and interobserver reliability. King classification demonstrated fair interobserver reliability and excellent intraobserver reliability. The mean interobserver reliability was 65.8% (Kappa coefficient = 0.542), while the intraobserver reliability was 82.6% (Kappa coefficient = 0.767). The main reason of disagreement was distinguishment of the King type II and type III. Another reason was assessment of King type V. The complete Lenke classification demonstrated poor reliability. The mean interobserver reliability was 50.0% (Kappa coefficient = 0.438), and the intraobserver reliability was 47.0% (Kappa coefficient = 0.402). The 3 components of Lenke classification had fair interobserver and intraobserver reliability when they were examined separately. The main reasons for disagreement arose from judging whether there was a structural upper thoracic curve and assigning sagittal thoracic modifier. The PUMC type demonstrated excellent intraobserver and interobserver reliability with a mean interobserver reliability of 87.8% (Kappa coefficient = 0.757), and a mean intraobserver reliability of 92.9% (Kappa coefficient = 0.958). Its subtypes demonstrated fair agreement with the mean interobserver reliability of 70.2% (Kappa coefficient = 0.629), and a mean intraobserver reliability of 74.1% (Kappa coefficient = 0.674). The main reasons for disagreement were definition of a curvature and Cobb angle measurement. King and PUMC classification systems have higher interobserver and intraobserver reliability than Lenke classification. The reliability levels of the 3 classification systems are all influenced by many factors. Judgment of an upper thoracic curve, variable of Cobb angle measurement, and relationship of the central vertical sagittal line to apex of curve are the common reasons for disagreement in these 3 classifications.