Interobserver Variation Research Articles

Apparatus and method for measuring individual color-matching functions

This article presents a compact visual colorimeter for the purpose of measuring color-matching functions (CMFs) of individual observers through psychophysical experiments. Constructed from 3D-printed parts, optical elements, and LED-based light engines, the colorimeter facilitates the juxtaposition of two fields to create a bipartite field. The system underwent characterization to evaluate factors that may impact color-matching experiments, such as LED-light stability, spatial homogeneity of the bipartite field, and potential stray-light leakage. The study aimed to assess the accuracy and performance of the system in measuring individual observer CMFs. Results indicate that the system is stable enough to measure both intra- and inter-observer variations in CMFs.

Three-dimensional source position verification in image-guided high-dose-rate brachytherapy using an XCT-based gel dosimeter.

Comprehensive quality assurance (QA) for a seamless workflow of high-dose-rate brachytherapy, from imaging to planning and irradiation, is uncommon, and QA of the source dwell position is performed in one- or two-dimensions. Gel dosimetry using magnetic resonance imaging (MRI) is effective in verifying the three-dimensional distribution of doses for image-guided brachytherapy (IGBT). However, MRI scanners are not readily accessible, and MRI scanning is time-consuming. Nevertheless, X-ray computed tomography (XCT) is available for IGBT planning. In this study, we designed and developed an efficient method for QA for a seamless workflow of IGBT with a new commercially available XCT-based polymer gel dosimeter. To enable direct insertion of brachytherapy applicators, the gelatinizing agent of the dosimeter was modified. A cylindrical polyvinyl chloride jar was filled with the modified gel dosimeter, which was subsequently used to determine the reproducibility of source dwell positions, detectability of source positional errors from intentionally introduced catheter length offsets, effect of looped source transfer tubes on the average displacement, extent of inter-observer variation, and gel robustness following multiple needle-insertions. Three ProGuide sharp needles were inserted into the jar. The dwell time at each point was determined to identify the irradiated volume with a diameter of approximately 10mm on XCT images. All the times were the same. The plan was delivered using an afterloader with an Ir-192 radioactive source, and the irradiated gel dosimeter was scanned using an XCT scanner. The subtracted images were generated from pre- and post-irradiated images. Volumes with incremented Hounsfield units were manually identified and contoured. The centroid of the volume was defined as the measured source dwell position. Subsequently, planned source dwell positions were extracted from the DICOM file of the plan. Finally, the source dwell positions in plan and irradiated gel were compared in three axes. The hardness of the dosimeter was 1250% greater than that of the previously reported gel dosimeter. Source dwell positions were visually identified in the XCT image. Testing of CT acquisition, planning, irradiation, and analysis was completed in approximately 1h. In the reproducibility test of source dwell positions, created by inserting three needles (each with three source dwell positions), the average displacements of the source positions from the first source dwell position were within 0.5mm in all three directions. In the detectability test, displacements were less than 1mm in the x-y plane but greater than 1mm in the z-axis, which was the source path direction. When errors of 1-3mm were intentionally introduced, the measured displacement was within 0.7mm of the median (range: 0.21-1.65mm) of intentional errors. When the transfer tube was looped, the source dwell position displaced by approximately 1mm. After 20 needle-insertions, the source dwell position displacement was within 1mm. The maximum inter-observer variation of contouring was 0.57mm. The XCT-based gel dosimeter enabled verification of three-dimensional source dwell positions for a seamless workflow of IGBT with high precision and efficiency.

Inter-observer variation in gross tumour volume delineation of oesophageal cancer on MR, CT and PET/CT.

The aim of our study was to assess the inter-observer variability in delineation of the gross tumour volume (GTV) of oesophageal cancer on magnetic resonance (MR) in comparison to computed tomography (CT) and positron emission tomography and CT (PET/CT). Twenty-three consecutive patients with oesophageal cancer treated with chemo-radiotherapy were enrolled. All patients had PET/CT and MR imaging in treatment position. Five observers independently delineated the GTV on CT alone, MR alone, CT with co-registered MR, PET/CT alone and MR with co-registered PET/CT. Volumes of GTV were measured per patient and imaging modality. Inter-observer agreement, expressed in generalized conformity index (CIgen), volumetric conformity index (VCI), planar conformity index (PCI) and inter-delineation distance (IDD) were calculated per patient and imaging modality. Linear mixed models were used for statistical analysis. GTV volume was significantly lower on MR (33.03 cm3) compared to CT (37.1 cm3; p = 0.002) and on PET/CT MR (35.2 cm3; p = 0.018) compared to PET/CT (39.1 cm3). The CIgen was lowest on CT (0.56) and highest on PET/CT MR (0.67). The difference in CIgen between MR (0.61) and CT was borderline significant (p = 0.048). The VCI was significantly higher on MR (0.71; p = 0.007) and on CT MR (0.71; p = 0.004) compared to CT (0.67). The PCI was significantly higher on CT MR (0.67; p = 0.031) compared to CT (0.64). The largest differences were observed in the cranio-caudal direction. The highest inter-observer agreement was found for PET/CT MR and the lowest for CT. MR could reduce the difference in delineation between observers and provide additional information about the local extent of the tumour.

On-site burn severity assessment using smartphone-captured color burn wound images

Accurate assessment of burn severity is crucial for the management of burn injuries. Currently, clinicians mainly rely on visual inspection to assess burns, characterized by notable inter-observer discrepancies. In this study, we introduce an innovative analysis platform using color burn wound images for automatic burn severity assessment. To do this, we propose a novel joint-task deep learning model, which is capable of simultaneously segmenting both burn regions and body parts, the two crucial components in calculating the percentage of total body surface area (%TBSA). Asymmetric attention mechanism is introduced, allowing attention guidance from the body part segmentation task to the burn region segmentation task. A user-friendly mobile application is developed to facilitate a fast assessment of burn severity at clinical settings. The proposed framework was evaluated on a dataset comprising 1340 color burn wound images captured on-site at clinical settings. The average Dice coefficients for burn depth segmentation and body part segmentation are 85.12 % and 85.36 %, respectively. The R2 for %TBSA assessment is 0.9136. The source codes for the joint-task framework and the application are released on Github (https://github.com/xjtu-mia/BurnAnalysis). The proposed platform holds the potential to be widely used at clinical settings to facilitate a fast and precise burn assessment.

Development and comprehensive evaluation of a national DBCG consensus-based auto-segmentation model for lymph node levels in breast cancer radiotherapy

Background and purposeThis study aimed at training and validating a multi-institutional deep learning (DL) auto segmentation model for nodal clinical target volume (CTVn) in high-risk breast cancer (BC) patients with both training and validation dataset created with multi-institutional participation, with the overall aim of national clinical implementation in Denmark. Materials and methodsA gold standard (GS) dataset and a high-quality training dataset were created by 21 BC delineation experts from all radiotherapy centres in Denmark. The delineations were created according to ESTRO consensus delineation guidelines. Four models were trained: One per laterality and extension of CTVn internal mammary nodes. The DL models were tested quantitatively in their own test-set and in relation to interobserver variation (IOV) in the GS dataset with geometrical metrics, such as the Dice Similarity Coefficient (DSC). A blinded qualitative evaluation was conducted with a national board, presented to both DL and manual delineations. ResultsA median DSC > 0.7 was found for all, except the CTVn interpectoral node in one of the models. In the qualitative evaluation ‘no corrections needed’ were acquired for 297 (36 %) in the DL structures and 286 (34 %) for manual delineations. A higher rate of ‘major corrections’ and ‘easier to start from scratch’ was found in the manual delineations. The models performed within the IOV of an expert group, with two exceptions. ConclusionDL models were developed on a national consensus cohort and performed on par with the IOV between BC experts and had a comparable or higher clinical acceptance than expert manual delineations.

Development and evaluation of machine-learning methods in whole-body magnetic resonance imaging with diffusion weighted imaging for staging of patients with cancer: the MALIBO diagnostic test accuracy study

Background Whole-body magnetic resonance imaging is accurate, efficient and cost-effective for cancer staging. Machine learning may support radiologists reading whole-body magnetic resonance imaging. Objectives To develop a machine-learning algorithm to detect normal organs and cancer lesions. To compare diagnostic accuracy, time and agreement of radiology reads to detect metastases using whole-body magnetic resonance imaging with concurrent machine learning (whole-body magnetic resonance imaging + machine learning) against standard whole-body magnetic resonance imaging (whole-body magnetic resonance imaging + standard deviation). Design and participants Retrospective analysis of (1) prospective single-centre study in healthy volunteers > 18 years (n = 51) and (2) prospective multicentre STREAMLINE study patient data (n = 438). Tests Index: whole-body magnetic resonance imaging + machine learning. Comparator: whole-body magnetic resonance imaging + standard deviation. Reference standard Previously established expert panel consensus reference at 12 months from diagnosis. Outcome measures Primary: difference in per-patient specificity between whole-body magnetic resonance imaging + machine learning and whole-body magnetic resonance imaging + standard deviation. Secondary: per-patient sensitivity, per-lesion sensitivity and specificity, read time and agreement. Methods Phase 1: classification forests, convolutional neural networks, and a multi-atlas approaches for organ segmentation. Phase 2/3: whole-body magnetic resonance imaging scans were allocated to Phase 2 (training = 226, validation = 45) and Phase 3 (testing = 193). Disease sites were manually labelled. The final algorithm was applied to 193 Phase 3 cases, generating probability heatmaps. Twenty-five radiologists (18 experienced, 7 inexperienced in whole-body magnetic resonance imaging) were randomly allocated whole-body magnetic resonance imaging + machine learning or whole-body magnetic resonance imaging + standard deviation over two or three rounds in a National Health Service setting. Read time was independently recorded. Results Phases 1 and 2: convolutional neural network had best Dice similarity coefficient, recall and precision measurements for healthy organ segmentation. Final algorithm used a ‘two-stage’ initial organ identification followed by lesion detection. Phase 3: evaluable scans (188/193, of which 50 had metastases from 117 colon, 71 lung cancer cases) were read between November 2019 and March 2020. For experienced readers, per-patient specificity for detection of metastases was 86.2% (whole-body magnetic resonance imaging + machine learning) and 87.7% (whole-body magnetic resonance imaging + standard deviation), (difference −1.5%, 95% confidence interval −6.4% to 3.5%; p = 0.387); per-patient sensitivity was 66.0% (whole-body magnetic resonance imaging + machine learning) and 70.0% (whole-body magnetic resonance imaging + standard deviation) (difference −4.0%, 95% confidence interval −13.5% to 5.5%; p = 0.344). For inexperienced readers (53 reads, 15 with metastases), per-patient specificity was 76.3% in both groups with sensitivities of 73.3% (whole-body magnetic resonance imaging + machine learning) and 60.0% (whole-body magnetic resonance imaging + standard deviation). Per-site specificity remained high within all sites; above 95% (experienced) or 90% (inexperienced). Per-site sensitivity was highly variable due to low number of lesions in each site. Reading time lowered under machine learning by 6.2% (95% confidence interval −22.8% to 10.0%). Read time was primarily influenced by read round with round 2 read times reduced by 32% (95% confidence interval 20.8% to 42.8%) overall with subsequent regression analysis showing a significant effect (p = 0.0281) by using machine learning in round 2 estimated as 286 seconds (or 11%) quicker. Interobserver variance for experienced readers suggests moderate agreement, Cohen’s κ = 0.64, 95% confidence interval 0.47 to 0.81 (whole-body magnetic resonance imaging + machine learning) and Cohen’s κ = 0.66, 95% confidence interval 0.47 to 0.81 (whole-body magnetic resonance imaging + standard deviation). Limitations Patient whole-body magnetic resonance imaging data were heterogeneous with relatively few metastatic lesions in a wide variety of locations, making training and testing difficult and hampering evaluation of sensitivity. Conclusions There was no difference in diagnostic accuracy for whole-body magnetic resonance imaging radiology reads with or without machine-learning support, although radiology read time may be slightly shortened using whole-body magnetic resonance imaging + machine learning. Future work Failure-case analysis to improve model training, automate lesion segmentation and transfer of machine-learning techniques to other tumour types and imaging modalities. Study registration This study is registered as ISRCTN23068310. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Efficacy and Mechanism Evaluation (EME) programme (NIHR award ref: 13/122/01) and is published in full in Efficacy and Mechanism Evaluation; Vol. 11, No. 15. See the NIHR Funding and Awards website for further award information.

Gastric-type extremely well-differentiated adenocarcinoma of the stomach: A rare tumor with diagnostic difficulties and high inter-observer variation in endoscopic pinch biopsies

Extremely well-differentiated gastric-type adenocarcinoma (EWDGA) is a rare type of gastric cancer composed of deceptively bland-looking malignant cells resembling normal foveolar or pyloric epithelium. The histological features of this tumor have not been recognized by many pathologists, and inter-observer variation studies are lacking. Here, we report seven EWDGAs and inter-observer variation of six preoperative biopsies was evaluated by 11 pathologists in a single institute. Based on the pathological diagnosis of the endoscopic biopsy slides, the average rate of definite malignancy diagnosis was 15.2 %, and the overall diagnostic concordance rate was 34.9 % among 11 pathologists. Microscopically, the surface epithelium was preserved and only a few atypical tumor glands were scattered in most endoscopic biopsies. Structural atypia was minimal, and the tumor glands were barely distinguishable from normal glands. Although nuclear atypia was minimal, enlarged nuclei, relatively large glands with irregular shapes, and abundant cytoplasmic mucin were observed in gastric pinch biopsies. In preoperative biopsies, no cases showed p53 overexpression, and Ki-67 labeling index ranged from 3 % to 35 % and was higher compared to non-neoplastic glands in 3 cases. After gastrectomy, four (57.1 %) patients had advanced gastric cancer and three (42.9 %) had lymph node metastasis. Genomic profiling of the four patients revealed mutations of TP53, BRAF, KRAS, STK11, and MDM2/CCND1 amplification. Immunohistochemistry for p53 was not helpful while Ki-67 may be helpful when staining pattern is distinct from the non-neoplastic mucosa. In conclusion, it is challenging to diagnose EWDGA using biopsy specimens. Recognizing and addressing this rare entity will increase diagnostic accuracy to ensure the early diagnosis of cancer.

Skeletal Muscle Segmentation at the Level of the Third Lumbar Vertebra (L3) in Low-Dose Computed Tomography: A Lightweight Algorithm.

The cross-sectional area of skeletal muscles at the level of the third lumbar vertebra (L3) measured from computed tomography (CT) images is an established imaging biomarker used to assess patients' nutritional status. With the increasing prevalence of low-dose CT scans in clinical practice, accurate and automated skeletal muscle segmentation at the L3 level in low-dose CT images has become an issue to address. This study proposed a lightweight algorithm for automated segmentation of skeletal muscles at the L3 level in low-dose CT images. This study included 57 patients with rectal cancer, with both low-dose plain and contrast-enhanced pelvic CT image series acquired using a radiotherapy CT scanner. A training set of 30 randomly selected patients was used to develop a lightweight segmentation algorithm, and the other 27 patients were used as the test set. A radiologist selected the most representative axial CT image at the L3 level for both the image series for all the patients, and three groups of observers manually annotated the skeletal muscles in the 54 CT images of the test set as the gold standard. The performance of the proposed algorithm was evaluated in terms of the Dice similarity coefficient (DSC), precision, recall, 95th percentile of the Hausdorff distance (HD95), and average surface distance (ASD). The running time of the proposed algorithm was recorded. An open source deep learning-based AutoMATICA algorithm was compared with the proposed algorithm. The inter-observer variations were also used as the reference. The DSC, precision, recall, HD95, ASD, and running time were 93.2 ± 1.9% (mean ± standard deviation), 96.7 ± 2.9%, 90.0 ± 2.9%, 4.8 ± 1.3 mm, 0.8 ± 0.2 mm, and 303 ± 43 ms (on CPU) for the proposed algorithm, and 94.1 ± 4.1%, 92.7 ± 5.5%, 95.7 ± 4.0%, 7.4 ± 5.7 mm, 0.9 ± 0.6 mm, and 448 ± 40 ms (on GPU) for AutoMATICA, respectively. The differences between the proposed algorithm and the inter-observer reference were 4.7%, 1.2%, 7.9%, 3.2 mm, and 0.6 mm, respectively, for the averaged DSC, precision, recall, HD95, and ASD. The proposed algorithm can be used to segment skeletal muscles at the L3 level in either the plain or enhanced low-dose CT images.

Comprehensive Clinical Usability-Oriented Contour Quality Evaluation for Deep Learning Auto-segmentation: Combining Multiple Quantitative Metrics Through Machine Learning

PurposeThe current commonly-used metrics for evaluating the quality of auto-segmented contours have limitations and do not always reflect the clinical usefulness of the contours. This work aims to develop a novel contour quality classification (CQC) method by combining multiple quantitative metrics for clinical usability-oriented contour quality evaluation for deep learning-based auto-segmentation (DLAS). MethodsThe CQC was designed to categorize contours on slices as acceptable, minor edit, or major edit based on the expected editing effort/time with supervised ensemble tree classification models using seven quantitative metrics. Organ-specific models were trained for five abdominal organs (pancreas, duodenum, stomach, small and large-bowels) using 50 MRI datasets. Twenty additional MRI and nine CT datasets were employed for testing. Inter-observer variation (IOV) was assessed among six observers and consensus labels were established through majority vote for evaluation. The CQC was also compared with a threshold-based baseline approach. ResultsFor the five organs, the average AUC was 0.982±0.01 and 0.979±0.01, the mean-accuracy was 95.8±1.7% and 94.3±2.1%, and the mean risk-rate was 0.8±0.4% and 0.7±0.5% for MRI and CT testing dataset, respectively. The CQC results closely matched the IOV results (mean-accuracy of 94.2±0.8% and 94.8±1.7%) and were significantly higher than those obtained using the threshold-based method (mean-accuracy of 80.0±4.7%, 83.8±5.2%, and 77.3±6.6% using one, two, and three metrics). ConclusionThe CQC models demonstrated high performance in classifying the quality of contour slices. This method can address the limitations of existing metrics and offers an intuitive and comprehensive solution for clinically oriented evaluation and comparison of DLAS systems.

Association of Oncotype-DX HER2 Single Gene Score With HER2 Expression Assessed by Immunohistochemistry in HER2-low Breast Cancer.

"HER2-low" is an emerging subtype of breast cancer, with a documented role in predicting response to treatment with novel antibody-drug conjugates. It is defined based on immunohistochemistry, but increasing evidence is challenging this approach as appropriate for identifying the HER2-low subgroup, due to both interobserver variability and limitations of the method itself. We retrospectively analyzed data from 430 patients from our departmental databases who had been subjected to an Oncotype-DX score and assessed the correlation of the Oncotype-DX HER2 single-gene score with the HER2 expression on immunohistochemistry. The Oncotype-DX Recurrence Score was also evaluated in the HER2-0 versus HER2-low subgroups. The HER2 single-gene score was found to accurately correlate with the HER2 result on immunohistochemistry, with a statistically significant difference both between HER2-0 and HER2 +1 tumors (p<0.0001), as well as between HER2 +1 and +2 tumors (p<0.0001). There was no statistically significant difference in the recurrence score between the HER2-0 and the HER2-low subgroups. Oncotype-DX single-gene scores for HER2 are a potential surrogate marker for assessing the precise HER2 status, with better reproducibility and less interobserver variance compared to immunohistochemistry. The use of rt-PCR emerges as an alternative method of assessment of the HER2-low subgroup.

2204: Motion management strategy and interobserver variation in abdominal OAR delineation using MRI

2204: Motion management strategy and interobserver variation in abdominal OAR delineation using MRI

Deep learning with uncertainty estimation for automatic tumor segmentation in PET/CT of head and neck cancers: impact of model complexity, image processing and augmentation

Objective. Target volumes for radiotherapy are usually contoured manually, which can be time-consuming and prone to inter- and intra-observer variability. Automatic contouring by convolutional neural networks (CNN) can be fast and consistent but may produce unrealistic contours or miss relevant structures. We evaluate approaches for increasing the quality and assessing the uncertainty of CNN-generated contours of head and neck cancers with PET/CT as input. Approach. Two patient cohorts with head and neck squamous cell carcinoma and baseline 18F-fluorodeoxyglucose positron emission tomography and computed tomography images (FDG-PET/CT) were collected retrospectively from two centers. The union of manual contours of the gross primary tumor and involved nodes was used to train CNN models for generating automatic contours. The impact of image preprocessing, image augmentation, transfer learning and CNN complexity, architecture, and dimension (2D or 3D) on model performance and generalizability across centers was evaluated. A Monte Carlo dropout technique was used to quantify and visualize the uncertainty of the automatic contours. Main results. CNN models provided contours with good overlap with the manually contoured ground truth (median Dice Similarity Coefficient: 0.75–0.77), consistent with reported inter-observer variations and previous auto-contouring studies. Image augmentation and model dimension, rather than model complexity, architecture, or advanced image preprocessing, had the largest impact on model performance and cross-center generalizability. Transfer learning on a limited number of patients from a separate center increased model generalizability without decreasing model performance on the original training cohort. High model uncertainty was associated with false positive and false negative voxels as well as low Dice coefficients. Significance. High quality automatic contours can be obtained using deep learning architectures that are not overly complex. Uncertainty estimation of the predicted contours shows potential for highlighting regions of the contour requiring manual revision or flagging segmentations requiring manual inspection and intervention.

Factors influencing the reliability of measurements in eyes with full-thickness macular holes: are we measuring incorrectly?

PurposeThe calliper function is used for manual measurements of full thickness macular holes (FTMHs). We aimed to investigate whether a reproducible difference could be detected beyond interobserver variability between two...

Agreement of Pain Assessment Using the Short Form of the Canine Glasgow Composite Measure Pain Scale between Veterinary Students, Veterinary Nurses, Veterinary Surgeons, and ECVAA-Diplomates.

Several pain scoring systems have been validated to measure pain in dogs. However, pain may not be adequately assessed since these tools are associated with high-level inter-observer variation. The aim of this study is to evaluate the agreement of pain assessment using the CMPS-SF between veterinary students, veterinary nurses, veterinary surgeons, and European College of Veterinary Anaesthesia and Analgesia (ECVAA) diplomates. Forty-five client-owned dogs presented to a teaching hospital were enrolled in this prospective, observational study. All dogs were pain-scored in vivo, while a video of the assessment was recorded and subsequently evaluated by twenty assessors, with five per group. Mean scores between groups were compared, and agreement within groups and agreement of the average scores between groups were assessed by calculating the intraclass correlation coefficient (ICC). The intervention point at which dogs were deemed to require additional analgesia was also evaluated. Overall agreement of pain assessment was poor (ICC = 0.494). Nurses had the best inter-observer agreement (ICC = 0.656), followed by ECVAA diplomates (ICC = 0.540), veterinary surgeons (ICC = 0.478), and veterinary students (ICC = 0.432). The best inter-group agreement was between veterinary surgeons and nurses (ICC = 0.951) and between ECVAA diplomates and nurses (ICC = 0.951). Students were more likely to determine that additional analgesia was required compared to other groups. Pain assessment is key for animal welfare, and training in this area should be reinforced to improve consistency.

Challenges with segmenting intraoperative ultrasound for brain tumours

Objective - Addressing the challenges that come with identifying and delineating brain tumours in intraoperative ultrasound. Our goal is to both qualitatively and quantitatively assess the interobserver variation, amongst experienced neuro-oncological intraoperative ultrasound users (neurosurgeons and neuroradiologists), in detecting and segmenting brain tumours on ultrasound. We then propose that, due to the inherent challenges of this task, annotation by localisation of the entire tumour mass with a bounding box could serve as an ancillary solution to segmentation for clinical training, encompassing margin uncertainty and the curation of large datasets. Methods - 30 ultrasound images of brain lesions in 30 patients were annotated by 4 annotators - 1 neuroradiologist and 3 neurosurgeons. The annotation variation of the 3 neurosurgeons was first measured, and then the annotations of each neurosurgeon were individually compared to the neuroradiologist’s, which served as a reference standard as their segmentations were further refined by cross-reference to the preoperative magnetic resonance imaging (MRI). The following statistical metrics were used: Intersection Over Union (IoU), Sørensen-Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD). These annotations were then converted into bounding boxes for the same evaluation. Results - There was a moderate level of interobserver variance between the neurosurgeons [IoU:0.789,DSC:0.876,HD:103.227]\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$[IoU:0.789, \\ DSC:0.876, \\ HD:103.227]$$\\end{document} and a larger level of variance when compared against the MRI-informed reference standard annotations by the neuroradiologist, mean across annotators [IoU:0.723,DSC:0.813,HD:115.675]\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$[IoU:0.723, \\ DSC:0.813, \\ HD:115.675]$$\\end{document}. After converting the segments to bounding boxes, all metrics improve, most significantly, the interquartile range drops by [IoU:37%,DSC:41%,HD:54%]\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$[IoU:37\\%, \\ DSC:41\\%, \\ HD:54\\%]$$\\end{document}. Conclusion - This study highlights the current challenges with detecting and defining tumour boundaries in neuro-oncological intraoperative brain ultrasound. We then show that bounding box annotation could serve as a useful complementary approach for both clinical and technical reasons.

Learning three-dimensional aortic root assessment based on sparse annotations.

Analyzing the anatomy of the aorta and left ventricular outflow tract (LVOT) is crucial for risk assessment and planning of transcatheter aortic valve implantation (TAVI). A comprehensive analysis of the aortic root and LVOT requires the extraction of the patient-individual anatomy via segmentation. Deep learning has shown good performance on various segmentation tasks. If this is formulated as a supervised problem, large amounts of annotated data are required for training. Therefore, minimizing the annotation complexity is desirable. We propose two-dimensional (2D) cross-sectional annotation and point cloud-based surface reconstruction to train a fully automatic 3D segmentation network for the aortic root and the LVOT. Our sparse annotation scheme enables easy and fast training data generation for tubular structures such as the aortic root. From the segmentation results, we derive clinically relevant parameters for TAVI planning. The proposed 2D cross-sectional annotation results in high inter-observer agreement [Dice similarity coefficient (DSC): 0.94]. The segmentation model achieves a DSC of 0.90 and an average surface distance of 0.96mm. Our approach achieves an aortic annulus maximum diameter difference between prediction and annotation of 0.45mm (inter-observer variance: 0.25mm). The presented approach facilitates reproducible annotations. The annotations allow for training accurate segmentation models of the aortic root and LVOT. The segmentation results facilitate reproducible and quantifiable measurements for TAVI planning.

Inter- and intra-observer variation in phytolith morphometry.

Archaeobotanists and palaeoecologists extensively use geometric morphometrics to identify plant opal phytoliths. Particularly when applied to assemblages of phytoliths from concentrations retrieved from closed contexts, morphometric data from archaeological phytoliths compared with similar data from reference material may allow taxonomic attribution. Observer variation is one aspect of phytolith morphometry that has received little attention but may be an important source of error, and hence cause of potential misidentification of plant remains. To investigate inter- and intra-observer variation in phytolith morphometry, eight researchers (observers) from different laboratories measured 50 samples each from three phytolith morphotypes, Bilobate, Bulliform flabellate and Elongate dendritic, three times, under the auspices of the International Committee for Phytolith Morphometrics (ICPM). Data for 17 size and shape variables were collected for each phytolith by manually digitising a phytolith outline (mask) from a photograph, followed by measurement of the mask with open-source morphometric software. Inter-observer variation ranged from 0 to 23% difference from the mean of all observers. Intra-observer variation ranged from 0 to 9% difference from the mean of individual observers per week. Inter- and intra-observer variation was generally higher among inexperienced researchers. Scaling errors were a major cause of variation and occurred more with less experienced researchers, which is likely related to familiarity with data collection. The results indicate that inter- and intra-observer variation can be substantially reduced by providing clear instructions for and training with the equipment, photo capturing, software, data collection and data cleaning. In this paper, the ICPM provides recommendations to minimise variation.Advances in automatic data collection may eventually reduce inter- and intra-observer variation, but until this is common practice, the ICPM recommends that phytolith morphometric analyses adhere to standardised guidelines to assure that measured phytolith variables are accurate, consistent and comparable between different researchers and laboratories.

Deep Residual Learning-Based Classification with Identification of Incorrect Predictions and Quantification of Cellularity and Nuclear Morphological Features in Digital Pathological Images of Common Astrocytic Tumors.

Interobserver variations in the pathology of common astrocytic tumors impact diagnosis and subsequent treatment decisions. This study leveraged a residual neural network-50 (ResNet-50) in digital pathological images of diffuse astrocytoma, anaplastic astrocytoma, and glioblastoma to recognize characteristic pathological features and perform classification at the patch and case levels with identification of incorrect predictions. In addition, cellularity and nuclear morphological features, including axis ratio, circularity, entropy, area, irregularity, and perimeter, were quantified via a hybrid task cascade (HTC) framework and compared between different characteristic pathological features with importance weighting. A total of 95 cases, including 15 cases of diffuse astrocytoma, 11 cases of anaplastic astrocytoma, and 69 cases of glioblastoma, were collected in Taiwan Hualien Tzu Chi Hospital from January 2000 to December 2021. The results revealed that an optimized ResNet-50 model could recognize characteristic pathological features at the patch level and assist in diagnosis at the case level with accuracies of 0.916 and 0.846, respectively. Incorrect predictions were mainly due to indistinguishable morphologic overlap between anaplastic astrocytoma and glioblastoma tumor cell area, zones of scant vascular lumen with compact endothelial cells in the glioblastoma microvascular proliferation area mimicking the glioblastoma tumor cell area, and certain regions in diffuse astrocytoma with too low cellularity being misrecognized as the glioblastoma necrosis area. Significant differences were observed in cellularity and each nuclear morphological feature among different characteristic pathological features. Furthermore, using the extreme gradient boosting (XGBoost) algorithm, we found that entropy was the most important feature for classification, followed by cellularity, area, circularity, axis ratio, perimeter, and irregularity. Identifying incorrect predictions provided valuable feedback to machine learning design to further enhance accuracy and reduce errors in classification. Moreover, quantifying cellularity and nuclear morphological features with importance weighting provided the basis for developing an innovative scoring system to achieve objective classification and precision diagnosis among common astrocytic tumors.

O-128 Can embryo re-expansion pattern post-biopsy for PGT-A have a predictive value on the outcome?

Abstract Study question Does re-expansion grade at 1 hour post-biopsy correlate with post-thawing re-expansion rate and live birth rate? Summary answer Embryo re-expansion 1-hour post-biopsy is associated with post-warming re-expansion and increased live birth. What is known already Blastocyst cryopreservation is a key component of PGT-A procedures. In non-PGT-A cycles, collapsing blastocysts before vitrification is commonly recommended to improve blastocyst viability. However, the procedure becomes more complex in PGT-A cycles.The European Society of Human Reproduction and Embryology (ESHRE) recommends an immediate freeze post-biopsy, although a definitive timeframe remains elusive. To date, no conclusive timing has been agreed upon. Some studies support prolonged culture post-biopsy (&amp;gt; 3hrs), citing improved implantation and pregnancy rates. Conversely, others advocate for rapid cryopreservation, endorsing an immediate approach within an hour, while mostly blastocysts are still collapsed due to the biopsy procedure. Study design, size, duration Retrospective study of 1141 single euploid blastocyst transfers (seFET) conducted between April 2021 and February 2023. Analysed factors included patients' age, BMI, embryo quality at biopsy and 1-hour post-biopsy (at vitrification), and the biopsy day. Live birth and Ongoing pregnancy (OPR), defined as viable pregnancy beyond 24 weeks, were assessed. Blastocysts were scored at a standardized time (1hr post TE-biopsy): 0 for complete collapse, 1 for starting re-expansion, and 2 for clear re-expanded cavity. Participants/materials, setting, methods PGT-A patients undergoing a seFET. Only cycles with embryos being biopsied on Days 5,6 or 7 and with a known pregnancy outcome were included. Main results and the role of chance Out of 1141 transfers, 461 transfers resulted in live birth (40.4%) and 108 were still ongoing above 24 weeks’ gestation at the time of last contact (9.4%) for a total OPR/LB rate of of 49.8%. One-hour post-biopsy 64.0% of blastocysts re-expanded to Grade 2, 29.3% to Grade 1, and 6.7% showed no re-expansion (Grade 0). The OPR was significantly different between embryos that re-expanded to grade 2, 1 or 0 (54.2%, 44.3% and 32.5%, respectively, P &amp;lt; 0.001). Regression analysis showed re-expanding to Grade 2 at 1-hour post-biopsy was significantly and independently associated with OPR/LB compared to embryos that did not re-expand (i.e., Grade 0) (aOR: 1.77, 95% CI: 1.01–3.12, P = 0.047), after adjusting for embryo quality metrics (ICM, TE grade, day of biopsy), body-mass index and endometrial preparation type. . Embryos with greater re-expansion 1-hour post-biopsy had higher re-expansion slopes post-thawing, indicating faster and greater re-expansion to the point of transfer. Results were significant for all trophectoderm categories (Grade A, B, and C; P = 0.016, &amp;lt;0.001 and &amp;lt;0.001, respectively). Moreover, re-expansion rate 1-hour post-biopsy was significantly and independently associated with post-thawing re-expansion slope after adjusting for ICM, TE, expansion grade and biopsy day (P &amp;lt; 0.001). Limitations, reasons for caution The re-expansion grading remains an in-house system that involves subjective assessment with a possible inter-observer variation. Wider implications of the findings This study clearly demonstrates that post-biopsy re-expansion pattern, obtained in a standardized approach, can be helpful for predicting live birth. It is a novel independent marker that is not reflected by ICM, TE quality or day of biopsy for selecting embryos for thawing and transfer. Trial registration number not applicable

Artificial intelligence-generated targets and inter-observer variation in online adaptive radiotherapy of bladder cancer

Background and purposeDaily target re-delineation in online adaptive radiotherapy (oART) introduces uncertainty. The aim of this study was to evaluate artificial intelligence (AI) generated contours and inter-observer target variation among radiotherapy technicians in cone-beam CT (CBCT) guided oART of bladder cancer. Materials and methodsFor each of 10 consecutive patients treated with oART for bladder cancer, one CBCT was randomly selected and retrospectively included. The bladder (CTV-T) was AI-segmented (CTV-TAI). Seven radiotherapy technicians independently reviewed and edited CTV-TAI, generating CTV-TADP. Contours were benchmarked against a ground truth contour (CTV-TGT) delineated blindly from scratch. CTV-TADP and CTV-TAI were compared to CTV-TGT using volume, dice similarity coefficient, and bidirectional local distance. Dose coverage (D99%>95 %) of CTV-TGT was evaluated for treatment plans optimized for CTV-TAI and CTV-TADP with clinical margins. Inter-observer variation among CTV-TADP was assessed using coefficient of variation and generalized conformity index. ResultsCTV-TGT ranged from 48.7 cm3 to 211.6 cm3. The median [range] volume difference was 4.5 [−17.8, 42.4] cm3 for CTV-TADP and −15.5 [−54.2, 4.3] cm3 for CTV-TAI, compared to CTV-TGT. Corresponding dice similarity coefficients were 0.87 [0.71, 0.95] and 0.84 [0.64, 0.95]. CTV-TGT was adequately covered in 68/70 plans optimized on CTV-TADP and in 6/10 plans optimized on CTV-TAI with clinical margins. The median [range] coefficient of variation was 0.08 [0.05, 0.11] and generalized conformity index was 0.78 [0.71, 0.88] among CTV-TADP. ConclusionsTarget re-delineation in CBCT-guided oART of bladder cancer demonstrated non-isotropic inter-observer variation. Manual adjustment of AI-generated contours was necessary to cover ground truth targets.