Consensus Scoring Research Articles

Near-infrared imaging in orthodontic intraoral scanners for early interproximal caries detection

Intraoral scanners commonly used in orthodontic offices now offer near-infrared imaging (NIRI) technology, advertised as a screening tool to identify interproximal caries. This study aimed to evaluate the reliability and validity of NIRI detection of interproximal carious lesions in a common intraoral scanner (iTero Element 5D; Align Technology, San Jose, Calif) with and without bitewing radiograph complement, compared with a microcomputed tomography (micro-CT) reference standard. Extracted human posterior teeth (premolars and molars) were selected for early (noncavitated) interproximal carious lesions (n= 39) and sound control surfaces (n= 47). The teeth were scanned via micro-CT for evaluation by 2 blinded evaluators using consensus scoring. The teeth were mounted to simulate anatomic interproximal contacts and underwent a NIRI scan using iTero Element 5D and bitewing radiographs. Two trained, calibrated examiners independently evaluated (1) near-infrared images alone with clinical photograph, (2) bitewing radiograph alone with clinical photograph, and (3) near-infrared images with bitewing radiograph and clinical photograph in combination, after at least a 10-day washout period between each evaluation. Interrater reliability was highest for NIRI alone (k= 0.533) compared with bitewing radiograph alone (k= 0.176) or in combination (k= 0.256). NIRI alone showed high specificity (0.83-0.96) and moderate sensitivity (0.42-0.63) compared with a micro-CT reference standard. Dentin lesions were significantly more reliably detected than enamel lesions. After rigorous training and calibration, NIRI can be used with moderate reliability, high specificity, and moderate sensitivity to detect noncavitated interproximal carious lesions.

Interrater Agreement of BT-RADS for Evaluation of Follow-up MRI in Patients with Treated Primary Brain Tumor.

The Brain Tumor Reporting and Data System (BT-RADS) is a structured radiology reporting algorithm that was introduced to provide uniformity in posttreatment primary brain tumor follow-up and reporting, but its interrater reliability (IRR) assessment has not been widely studied. Our goal is to evaluate the IRR among neuroradiologists and radiology residents in the use of BT-RADS. This retrospective study reviewed 103 consecutive MR studies in 98 adult patients previously diagnosed with and treated for primary brain tumor (January 2019 to February 2019). Six readers with varied experience (4 neuroradiologists and 2 radiology residents) independently evaluated each case and assigned a BT-RADS score. Readers were blinded to the original score reports and the reports from other readers. Cases in which at least 1 neuroradiologist scored differently were subjected to consensus scoring. After the study, a post hoc reference score was also assigned by 2 readers by using future imaging and clinical information previously unavailable to readers. The interrater reliabilities were assessed by using the Gwet AC2 index with ordinal weights and percent agreement. Of the 98 patients evaluated (median age, 53 years; interquartile range, 41-66 years), 53% were men. The most common tumor type was astrocytoma (77%) of which 56% were grade 4 glioblastoma. Gwet index for interrater reliability among all 6 readers was 0.83 (95% CI: 0.78-0.87). The Gwet index for the neuroradiologists' group (0.84 [95% CI: 0.79-0.89]) was not statistically different from that for the residents' group (0.79 [95% CI: 0.72-0.86]) (χ2 = 0.85; P = .36). All 4 neuroradiologists agreed on the same BT-RADS score in 57 of the 103 studies, 3 neuroradiologists agreed in 21 of the 103 studies, and 2 neuroradiologists agreed in 21 of the 103 studies. Percent agreement between neuroradiologist blinded scores and post hoc reference scores ranged from 41%-52%. A very good interrater agreement was found when tumor reports were interpreted by independent blinded readers by using BT-RADS criteria. Further study is needed to determine if this high overall agreement can translate into greater consistency in clinical care.

Interpretation of PD-L1 expression in gastric cancer: summary of a consensus meeting of Korean gastrointestinal pathologists.

Nivolumab plus chemotherapy in the first-line setting has demonstrated clinical efficacy in patients with human epidermal growth factor receptor 2-negative advanced or metastatic gastric cancer, and is currently indicated as a standard treatment. Programmed death-ligand 1 (PD-L1) expression is an important biomarker for predicting response to anti-programmed death 1/PD-L1 agents in several solid tumors, including gastric cancer. In the CheckMate-649 trial, significant clinical improvements were observed in patients with PD-L1 combined positive score (CPS) ≥ 5, determined using the 28-8 pharmDx assay. Accordingly, an accurate interpretation of PD-L1 CPS, especially at a cutoff of 5, is important. The CPS method evaluates both immune and tumor cells and provides a comprehensive assessment of PD-L1 expression in the tumor microenvironment of gastric cancer. However, CPS evaluation has several limitations, one of which is poor interobserver concordance among pathologists. Despite these limitations, clinical indications relying on PD-L1 CPS are increasing. In response, Korean gastrointestinal pathologists held a consensus meeting for the interpretation of PD-L1 CPS in gastric cancer. Eleven pathologists reviewed 20 PD-L1 slides with a CPS cutoff close to 5, stained with the 28-8 pharmDx assay, and determined the consensus scores. The issues observed in discrepant cases were discussed. In this review, we present cases of gastric cancer with consensus PD-L1 CPS. In addition, we briefly touch upon current practices and clinical issues associated with assays used for the assessment of PD-L1 expression in gastric cancer.

Accuracy of a chatbot in answering questions that patients should ask before taking a new medication

BackgroundThe potential uses of artificial intelligence have extended into the fields of health care delivery and education. However, challenges are associated with introducing innovative technologies into health care, particularly with respect to information quality. ObjectiveThis study aimed to evaluate the accuracy of answers provided by a chatbot in response to questions that patients should ask before taking a new medication. MethodsTwelve questions obtained from the Agency for Healthcare Research and Quality were asked to a chatbot for the top 20 drugs. Two reviewers independently evaluated and rated each response on a 6-point scale for accuracy and a 3-point scale for completeness with a score of 2 considered adequate. Accuracy was determined using clinical expertise and a drug information database. After the independent reviews, answers were compared, and discrepancies were assigned a consensus score. ResultsOf 240 responses, 222 (92.5%) were assessed as completely accurate. Of the inaccurate responses, 10 (4.2%) were mostly accurate, 5 (2.1%) were more accurate than inaccurate, 2 (0.8%) were equal parts accurate and inaccurate, and 1 (0.4%) was more inaccurate than accurate. Of the 240 responses, 194 (80.8%) were comprehensively complete. There were 235 (97.9%) responses that scored 2 or higher. Five responses (2.1%) were considered incomplete. ConclusionUsing a chatbot to answer questions commonly asked by patients is mostly accurate but may include inaccurate information or lack valuable information for patients.

Are Generative Pretrained Transformer 4 Responses to Developmental Dysplasia of the Hip Clinical Scenarios Universal? An International Review.

There is increasing interest in applying artificial intelligence chatbots like generative pretrained transformer 4 (GPT-4) in the medical field. This study aimed to explore the universality of GPT-4 responses to simulated clinical scenarios of developmental dysplasia of the hip (DDH) across diverse global settings. Seventeen international experts with more than 15 years of experience in pediatric orthopaedics were selected for the evaluation panel. Eight simulated DDH clinical scenarios were created, covering 4 key areas: (1) initial evaluation and diagnosis, (2) initial examination and treatment, (3) nursing care and follow-up, and (4) prognosis and rehabilitation planning. Each scenario was completed independently in a new GPT-4 session. Interrater reliability was assessed using Fleiss kappa, and the quality, relevance, and applicability of GPT-4 responses were analyzed using median scores and interquartile ranges. Following scoring, experts met in ZOOM sessions to generate Regional Consensus Assessment Scores, which were intended to represent a consistent regional assessment of the use of the GPT-4 in pediatric orthopaedic care. GPT-4's responses to the 8 clinical DDH scenarios received performance scores ranging from 44.3% to 98.9% of the 88-point maximum. The Fleiss kappa statistic of 0.113 ( P = 0.001) indicated low agreement among experts in their ratings. When assessing the responses' quality, relevance, and applicability, the median scores were 3, with interquartile ranges of 3 to 4, 3 to 4, and 2 to 3, respectively. Significant differences were noted in the prognosis and rehabilitation domain scores ( P < 0.05 for all). Regional consensus scores were 75 for Africa, 74 for Asia, 73 for India, 80 for Europe, and 65 for North America, with the Kruskal-Wallis test highlighting significant disparities between these regions ( P = 0.034). This study demonstrates the promise of GPT-4 in pediatric orthopaedic care, particularly in supporting preliminary DDH assessments and guiding treatment strategies for specialist care. However, effective integration of GPT-4 into clinical practice will require adaptation to specific regional health care contexts, highlighting the importance of a nuanced approach to health technology adaptation. Level IV.

Development of a Value Assessment Framework for Pediatric Health Technologies Using Multicriteria Decision Analysis: Expanding the Value Lens for Funding Decision Making

ObjectivesA health technology assessment (HTA) does not systematically account for the circumstances and needs of children and youth. To supplement HTA processes, we aimed to develop a child-tailored value assessment framework using a multicriteria decision analysis approach. MethodsWe constructed a multicriteria-decision-analysis-based model in multiple phases to create the Comprehensive Assessment of Technologies for Child Health (CATCH) framework. Using a modified Delphi process with stakeholders having broad disciplinary and geographic variation (N = 23), we refined previously generated criteria and developed rank-based weights. We established a criterion-pertinent scoring rubric for assessing incremental benefits of new drugs. Three clinicians independently assessed comprehension by pilotscoring 9 drugs. We then validated CATCH for 2 childhood cancer therapies through structured deliberation with an expert panel (N = 10), obtaining individual scores, consensus scores, and verbal feedback. Analyses included descriptive statistics, thematic analysis, exploratory disagreement indices, and sensitivity analysis. ResultsThe modified Delphi process yielded 10 criteria, based on absolute importance/relevance and agreed importance (median disagreement indices = 0.34): Effectiveness, Child-specific Health-related Quality of Life, Disease Severity, Unmet Need, Therapeutic Safety, Equity, Family Impacts, Life-course Development, Rarity, and Fair Share of Life. Pilot scoring resulted in adjusted criteria definitions and more precise score-scaling guidelines. Validation panelists endorsed the framework’s key modifiers of value. Modes of their individual prescores aligned closely with deliberative consensus scores. ConclusionsWe iteratively developed a value assessment framework that captures dimensions of child-specific health and nonhealth gains. CATCH could improve the richness and relevance of HTA decision making for children in Canada and comparable health systems.

Abstract 4945: Computational modeling-based discovery of novel anticancer drug candidates targeting centromere-associated protein E

Abstract Purpose: Centromere-associated protein E (CENP-E), a mitotic motor protein, is crucial for cell division. Inhibition of CENP-E can lead to chromosome misalignment and aneuploidy. Our recent studies revealed that the CENP-E inhibitor (CENP-Ei), Compound A, induces aneuploid-mediated cGAS-STING pathway activation in cancer cell lines, which is expected to trigger innate immune response and induce immunological conversion of the tumor microenvironment from cold to hot. However, known CENP-Eis exhibited limited efficacy, and none have gone beyond phase I trials since 2012. The aim of this study is to develop a workflow integrating structure and artificial intelligence-based modeling approaches to accelerate the discovery of novel CENP-Eis with improved efficacy and overcome related challenges. Method: The inhibitor-bound CENP-E structure was modeled using chimeric homology modeling followed by induced fit docking with known CENP-Eis. Structure-based virtual screening (VS) was performed on large compound libraries in Enamine REAL, Synthetically Accessible Virtual Inventory, and MolPort databases. Consensus scoring methods were applied by averaging ranks of individual molecules obtained from VS against different CENP-E conformations. An active learning-enhanced VS pipeline was developed using ATOM Modeling PipeLine (AMPL) to find the best-ranked ligands more efficiently. CENP-E ligand candidates were further filtered based on their safety properties predicted by machine learning models trained with curated databases of adverse drug reactions using AMPL. The success of this study was measured by its ability to reveal novel chemotypes that could modulate CENP-E in kinesin ATPase assays. Result: Conformations of CENP-E’s ligand binding site were predicted for VS using the homology model. Structure-based VS was performed to select high-ranked compounds based on consensus scoring. The active learning-enhanced VS pipeline achieved 75% accuracy in ranking the top 20% of hit compounds. The graph convolutional network model with a ROCAUC, 0.76, was created using AMPL to filter ligand candidates based on their predicted safety properties. Seventy-three potential CENP-E ligands computationally prioritized from the Molport database were experimentally tested using kinesin ATPase assays from which 4 potential inhibitors and 8 potential activators were identified indicating overlapping binding sites of inhibitors and activators. Conclusion: Our integrated workflow is effective in discovering novel CENP-E modulators. The discovery of CENP-E activators provides an opportunity to identify novel therapeutics in hepatocellular carcinoma malignancies known to experience decreased CENP-E activity leading to aneuploidy. The predictions generated by computational models combined with experimental validations could further accelerate anticancer drug discovery. Citation Format: Pinyi Lu, Ryo Kamata, Michael R. Weil, Naomi Ohashi, Akihiro Ohashi, Eric A. Stahlberg. Computational modeling-based discovery of novel anticancer drug candidates targeting centromere-associated protein E [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4945.

Abstract 4587: Development of highly selective, potent, orally available PIM1 inhibitor BLX0631 shows a therapeutics potential in multiple myeloma models

Abstract Background: The Proto-oncogenic PIMs family are nuclear and cytoplasmic Ser/Thr kinases (PIM1, PIM2, and PIM3) that are constitutively active and play a vital role in proliferation and survival in Multiple Myeloma (MM). Activated PIM1 kinase can induce progression of the cell cycle, inhibition of apoptosis, and modulation of other signal transduction pathways. Methods: BLX-0631 and a few analogs evaluated against a panel of MM cell lines (EJM, IM-9, L-363, LP-1, MM-1R, MM.1S, MOLP-2, NCI-H929, OPM-2, RPMI8226 and U-266) and primary MM patient samples by CellTiter-Glo® (CTG) assays. Cellular apoptosis, Cell cycle analysis, Flow Cytometry, and bone marrow microenvironment experiments on the most sensitive cells were performed. PIM1 and downstream targets by western blot, cell migration, and invasion on our lead PIM1 inhibitor BLX-0631 along with, kinome selectivity, ADME-Tox, PK, and in vivo MM.1S efficacy experiments were conducted. Results: The application of our empirical MolecuLernTM fragment library, PIM1 crystal structure, scaffold hopping, and the enumeration features of MolecuLernTM in combination with 3, 6, and 7-group analysis on pyridazine, lead to a synthesizable potential new lead series, representing Med Chem tractability and developable characteristics. Further, the consensus scoring, binding energies, and MolecuLern tractability filters were applied and led to the discovery of the BLX-0631 series that inhibits the PIM1 kinase. The active compounds, with their binding modes and lead optimization strategies, lead to the synthesis and testing of over 60 novel entities as PIM1 kinase inhibitors. Thus, based on our previous chemotype series and scaffold hopping method, the impact of changing the 3, 6, and 7 positions, while retaining the pyridazine core led to the discovery of BLX-0631 and its series of compounds exhibiting &amp;lt;5 nM potency. BLX-0631 in CTG assay demonstrated 100% cell growth inhibition across all MM cell lines tested. The lead compound BLX-0631 is a very potent inhibitor with an IC50 of 5.0/17/5.0 nM when tested against PIM1, PIM2, and PIM3 kinases respectively, and selective against a panel of 468 kinase panels. Dosing through IV and PO delivery routes, formulations, salt forms of BLX-0631 in mice, PK properties along with ADME-Tox, P450, and hERG results will be presented. In vivo MM.1S mouse xenograft models of BLX-0631 at 15, 30 and 45 mg/Kg, p.o, QD demonstrated TGI of 16.28%, 31.81% &amp; 35.16% dose-dependently. Conclusions: The present study demonstrates that PIM1 plays a vital role in the proliferation and survival of Multiple Myeloma. BLX-0631 induces full inhibition of all MM cell lines, suppresses cancer cell proliferation in vitro, and exhibits promising efficacy in MM tumor models. BLX-0631, a nominated IND candidate can serve as a novel targeted agent for treating Multiple Myeloma patients. Citation Format: Zhaoang li, Chenyu Lin, David J. Bearss, Hariprasad Vankayalapati. Development of highly selective, potent, orally available PIM1 inhibitor BLX0631 shows a therapeutics potential in multiple myeloma models [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4587.

Abstract 4908: Comprehensive tool for pan-cancer consensus data mining of drugs, targets and biomarkers

Abstract The rapid increase in collateral omics and phenotypic data has enabled data-driven studies for the discovery of drug targets and biomarkers. For this purpose, we have developed a convenient tool, Q-omics software, for general researchers and cancer scientists to carry out customized data mining without bioinformatics background. Q-omics provides a comprehensive interface and smart functions for facilitating pan cancer-wide association studies on mutations, RNA expression, protein abundance, shRNA/CRISPR efficacy, drugs, tumor-infiltrating cells, patient survival, clinical info. etc. Furthermore, consensus scores on associated data pairs from heterogeneous samples, help to predict the reproducibility of data mining results. Q-omics (download: http://qomics.io) improves the utility of cancer omics big data for non-computational scientists at all levels of cancer research. Citation Format: Sukjoon Yoon. Comprehensive tool for pan-cancer consensus data mining of drugs, targets and biomarkers [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4908.

Artificial intelligence algorithm accurately assesses oestrogen receptor immunohistochemistry in metastatic breast cancer cytology specimens: A pilot study.

The Visiopharm artificial intelligence (AI) algorithm for oestrogen receptor (ER) immunohistochemistry (IHC) in whole slide images (WSIs) has been successfully validated in surgical pathology. This study aimed to assess its efficacy in cytology specimens. The study cohort comprised 105 consecutive cytology specimens with metastatic breast carcinoma. ER IHC WSIs were seamlessly integrated into the Visiopharm platform from the Image Management System (IMS) during our routine digital workflow, and an AI algorithm was employed for analysis. ER AI scores were compared with pathologists' manual consensus scores. Optimization steps were implemented and evaluated to reduce discordance. The overall concordance between pathologists' scores and AI scores was excellent (99/105, 94.3%). Six cases exhibited discordant results, including two false-negative (FN) cases due to abundant histiocytes incorrectly counted as negatively stained tumour cells by AI, two FN cases owing to weak staining, and two false-positive (FP) cases where pigmented macrophages were erroneously counted as positively stained tumour cells by AI. The Pearson correlation coefficient of ER-positive percentages between pathologists' and AI scores was 0.8483. Optimization steps, such as lowering the cut-off threshold and additional training using higher input magnification, significantly improved accuracy. The automated ER AI algorithm demonstrated excellent concordance with pathologists' assessments and accurately differentiated ER-positive from ER-negative metastatic breast carcinoma cytology cases. However, precision in identifying tumour cells in cytology specimens requires further enhancement.

Abstract 6182: Artificial intelligence (AI)-powered quantification of glypican-3 (GPC3) expression facilitates patient selection for GPC3-targeted therapy in solid tumors

Abstract Introduction: GPC3 is a glycosyl-phosphatidylinositol-anchored cell surface glycoprotein that functions to control cell growth and differentiation in early development. Membrane GPC3 is highly expressed in hepatocellular carcinoma (HCC) and moderately in squamous non-small cell lung cancer (SQ-NSCLC) with limited expression in normal tissues which makes it an attractive target for immune cell engagers. This study aims to develop an AI-powered algorithm to evaluate GPC3 expression and compare it with manual immunohistochemistry (IHC) scoring in HCC, adeno-NSCLC and SQ-NSCLC. Methods: 147 NSCLC (94 SQ-NSCLC and 53 adeno-NSCLC) and 113 HCC FFPE tumor blocks were obtained from commercial sources. GPC3 protein level was quantified by a qualified IHC assay for tumor cell membrane and cytoplasm individually. PD-L1 expression is reported as tumor cell proportion score (TPS), combined positive score (CPS) and immune cell proportion score (IPS/IC) TPS, regardless of staining pattern and intensity. FFPE slides were stained for GPC3+ (GC33, Ventana), PD-L1+ (22C3, Dako) cells and then scanned at 20x using the Aperio Versa8 scanner. Machine learning (ML) models were developed using the digitized whole slide images to identify GPC3+ tumor area. Data-driven cutoffs were applied to model-generated human-interpretable features of GPC3+ percentage of tumor area to classify samples as positive and negative. As an orthogonal approach to the GPC3+ percentage cutoff method, tissue and cell annotations were used to train a convolutional neural network (CNN) to classify GPC3 positivity. Results: The epidemiology profiling results revealed that GPC3 protein is highly expressed in HCC (57.5% cases with &amp;gt; 1% positive cells), followed by SQ-NSCLC (52.1%) and adeno-NSCLC (5.7%). GPC3 expression was only detected in tumor cells with two staining patterns observed: membrane + cytoplasm and cytoplasm-only. There was no significant correlation between GPC3 and PD-L1 expression across tumor types, although a trend of lower level of PD-L1 was observed in the samples with GPC3 ≥50%. ML model quantification of GPC3+ and GPC3- showed high concordance with the consensus score of 2 independent pathologists on a test set of held-out samples not used in training. This concordance was comparable to that of individual annotators to consensus. Instead, GPC3 positivity results using both cutoff and CNN methods showed moderate concordance with 2-way pathologist consensus. Conclusions: Here we profiled GPC3 expression across tumor types and showed high level of expression in HCC followed by SQ-NSCLC. We have established an AI-powered digital pathology platform that can provide a standardized, scalable, and reproducible method of characterizing GPC3 positivity to support further patient selection in clinical study. Citation Format: Ellen Meng, Robin Meng, Ying Tong, Yuchen Li, Hong Wang, Paola Fiorentini, Elham Attieh, Qi Tang, Asma Kefsi, Serena Masciari, Giovanni Abbadessa, Cecile Combeau, Lei Tang, Benoit Pasquier, Rui Wang. Artificial intelligence (AI)-powered quantification of glypican-3 (GPC3) expression facilitates patient selection for GPC3-targeted therapy in solid tumors [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 6182.

Supporting K-8 Teachers’ Capacity to Develop Quality Formative Science Assessments

ABSTRACT During a two-year span, a cohort of K-8 science teachers undertook the task of creating formative assessments aligned with the Next-Generation Science Standards for their individual classrooms. Practice-based professional development was utilized to support teachers in a cyclical manner as they cooperatively developed, administered, and reflected on a series of assessments they designed across two study phases. In order to measure developmental progression of the formative assessments across the study’s two phases, project leadership designed an assessment quality rating instrument targeting four components of a given formative assessment: (a) learning goal(s), (b) task/prompt, (c) success criteria, and (d) next instructional steps. Using this criterion-based instrument, 123 assessments were subjected to consensus scoring using a 0–3 rating for each component (e.g. learning goals). Results indicated teachers developed higher quality formative assessments during the project’s second phase. More specifically, they developed learning goals and success criteria that were identified as being multi-dimensional, meaning the assessment component was designed to assess more than content. Implications for practitioners and researchers are discussed, specifically related to the supports needed for teachers learning to progressively design and implement full spectrum, science-based, formative assessments. Finally, suggestions for future research are discussed given the potential value of using assessment design to introduce a cycle of teacher-driven self-reflection.

Comparison of consistency in acute stroke: Automated ASPECT score software tools and manual consensus scores

PurposeThe Alberta Stroke Program Early Computed Tomography Score (ASPECTS) is a standardized semi-quantitative scoring system. It is widely used to assess the extent of acute ischemic stroke (AIS). We evaluated the consistency of three automatic software packages with the overall and region-specific manual ASPECTS scores of AIS patients. MethodsRetrospectively, we gathered patients who presented with stroke symptoms between February 2019 and June 2022, and 174 cases were eventually included in the trial. Two radiologists reviewed the NCCT images independently; After four weeks, the same two radiologists began randomly reviewing the DWI images, discussed different scores and give consistent results as ground truth. ResultsMedian ASPECTS of the expert consensus reading was 7 (5–9). Good to excellent correlation of ASPECTS total scores among the three software tools (0.70, 0.74 and 0.83). Correlation among ground truth and Rapid-ASPECTS, RealNow-ASPECTS, ShuKun-ASPECTS (ICC = 0.51, Cronbach's α = 0.53), (ICC = 0.60, Cronbach's α = 0.70) and (ICC = 0.52, Cronbach's α = 0.64) respectively. The AUCs for Rapid, RealNow and ShuKun were 0.61, 0.67, and 0.62 respectively. The region-specific results showed a poor to good correlation. The correlations between the non-dominant and dominant cerebral hemispheres and the ground truth were statistically different (P < 0.05). ConclusionsOverall, the scoring consistency between the three automated scoring software and the ground truth is comparable, with RealNow-ASPECTS being no less consistent and effective than Rapid-ASPECTS and ShuKun-ASPECTS, and even better than both. But the consistency grade that still is developable.

Evaluating Hymenoptera Venom Allergy Severity: A Data-Centric Comparison of Grading Instruments

Introduction: While a consensus seems to have been reached with regard to the definition of anaphylaxis, there is no universal instrument for scoring allergic reaction severity despite more than 30 having been proposed by the time of writing. This severely hampers comparison of data between studies. While scales have been compared with regard to their utility in grading food-related reactions, no such comparisons have been made for Hymenoptera venom-associated reactions. Methods: The study conducted a retrospective analysis to compare the severity of Hymenoptera venom allergy reactions in 104 participants with suspected Hymenoptera venom allergy. The study applied six grading instruments to each reaction, also evaluating them against the NIAID/FAAN anaphylaxis criteria. Sensitivity, specificity, and receiver operating characteristic area under the curve (AUC) for identifying anaphylaxis were calculated. Severity scales were simplified into “mild,” “moderate,” and “severe” categories. The most common severity grade across the five scales was determined using a custom function to establish a consensus severity grade. Results: The most common culprit insects were honeybees (49.0%). Among the 88 participants with generalized reactions, the highest proportion had involvement of four organ systems. The scales showed high specificity for detecting anaphylaxis, especially when using higher grades of the Mueller, WAO, and Dribin scales. The diagnostic yields (AUC) varied, with the WAO scale having the highest AUC (0.94) for grades 3, 4, and 5. Spearman correlation analysis showed the strongest correlations seen between the Brown and Dribin, Ring and Messmer and Dribin, and Ring and Messmer and Reisman scales. The lowest correlations were observed with the Mueller scale when paired with the WAO, Reisman, and Dribin scales. An inter-rater reliability analysis showed substantial agreement between scales with the same number of grading levels. The agreement was highest for the Brown and Dribin scales, indicating a strong consistency in reaction severity classification across different instruments. Conclusion: While all instruments were effective in stratifying reactions, they showed limitations in differentiating milder phenotypes. The Brown and Dribin scales stood out for their high agreement with the consensus score and sensitivity in identifying anaphylaxis. Our findings suggest that adopting either of these scales could significantly unify the reporting of allergic reactions. We believe the format of an instrument should be tailored to its intended purpose, with clinical decision aids being simpler and research tools being more detailed.

Quality appraisal of clinical practice guidelines addressing massage interventions using the AGREE II instrument

ObjectiveThe purpose of this study was to systematically evaluate the methodological quality of massage-related clinical practice guidelines (CPGs)/consensus on massage using the Appraisal of Guidelines Research and Evaluation (AGREE II) instrument and to summarize the current status of recommendations in the CPGs.MethodsThe Chinese National Knowledge Infrastructure (CNKI), WanFang Data, China Science and Technology Journal Database (VIP), China Biology Medicine disc (CBM), PubMed, Embase, and guideline websites (such as the Chinese Medical Ace Base, the China Association of Chinese Medicine, the World Health Organization, Guideline International Network, National Institute for Health and Care Excellence, Scottish Intercollegiate Guidelines Network) were searched from inception to October 31, 2022. In addition, the reference lists of relevant studies were reviewed to identify domestic and overseas massage CPGs/consensus. The search terms adopted a combination of subject words and free words, mainly including traditional Chinese medicine, complementary therapies, Tuina, massage, manipulation, chiropractic/osteopathic, spinal, acupressure, guideline, and consensus. Two researchers independently completed the eligible records and extracted the data. Before the formal research, calibrations were performed twice on AGREE II, and all reviewers completed the pilot test three times until they understood and reached an agreement on the assessment items. Three researchers appraised the methodological quality of the included guidelines using the AGREE II instrument and calculated the overall intraclass correlation coefficient (ICC) of agreement.ResultsThe evaluation results showed that among the 49 eligible CPGs/consensus, 4 (8.2%) CPGs/consensus were considered “recommended”, 15 (30.6%) CPGs/consensus were considered “recommended with modifications”, and 30 (61.2%) CPGs/consensus were considered “not recommended”, while the consensus was considered “not recommended”. Generally, the scores in the six domains of the guidelines were all higher than the consensus. Evaluation results for the overall quality of 36 CPGs showed that 4 (11%) were “good quality”, 15 (42%) were “sufficient quality” and 17 (47%) were “lower quality”. The AGREE II quality scores of domains ranged from 0.30 to 0.75 ([ICC = 0.993, 95% CI (0.992, 0.995)]). The domain of scope and purpose (domain 1), with a median score of 0.75 (0.52~0.91), performed best in the guidelines with AGREE II, and stakeholder involvement (domain 2) [median 0.39 (0.31~0.56)] and application (domain 5) [median 0.30 (0.17~0.47] obtained lower scores. The consensus score of domain 1 was better at 26.0 (21.6~44.8), followed by rigor of development (domain 3) with a score of 18.0 (10.0~28.9). A total of 119 massage-related recommendations were extracted from 49 guidelines/consensuses, including “in favor” (102, 85.7%), “against” (9, 7.6%), and “did not make recommendations” (8, 6.7%).ConclusionThe overall quality of the included guidelines was low, and most of the guidelines were not “recommended”. In future guideline updates, the existing evidence should be used, the professional composition of members of the expert group should be enriched, and patients’ values and preferences should be fully considered. It is necessary to clearly propose recognizable recommendations and strengthen the rigor and standardization of guideline formulation. Thus, clear standard guidelines can be formulated to better guide clinical practice.

Concordance between pathologists and between specimen types in detection of HER2-low breast carcinoma by immunohistochemistry

ContextRecent clinical trials indicate that HER2-targeted therapy may benefit HER2-low breast cancer patients including HER2 score 1+ or 2+ and no gene amplification.Concordance between pathologists and between core biopsy and surgical excision in establishing HER2-low status was evaluated. Design57 patients with HER2 negative breast cancer (IHC 0, 1+, or 2+, no gene amplification) by core biopsy were included. Core biopsy and representative tumor from corresponding surgical excision was immunostained for HER2. Original HER2 IHC scores were interpreted using 2018 guidelines. Three pathologists independently interpreted again under 2023 guidelines. Kappa statistic evaluated agreement of HER2 IHC scores. ResultsApplying 2023 guidelines, HER2 IHC scores were concordant among study pathologists in 46 of 57 (81 %) core biopsy and 50 of 57 (88 %) surgical resections. Kappa statistics were 0.78 and 0.85 (substantial agreement), for inter-pathologist agreement of core biopsy and surgical resections under 2023 guidelines; 0.55 (moderate agreement) for agreement between first interpretation by 2018 guidelines and second interpretation by 2023 guidelines; and 0.13 (slight agreement) for agreement in HER2 consensus scores between outside core and surgical resection and 0.49 (moderate agreement) for inside core and surgical resection. Low HER2 expression was found in 28 of 57 (49 %) core biopsy and in 25 of 57 (44 %) surgical excisions. ConclusionsInterobserver agreement among study pathologists was good in core biopsy and surgical excisions, applying updated 2023 guidelines. Intratumoral heterogeneity in protein expression and preanalytical factors may result in variable identification of HER2-low status in core biopsy and surgical excision specimens.

Applicability of the Barcelona scale to assess the quality of cleanliness of mucosa at esophagogastroduodenoscopy

Background and objectivesThere are few scales with prospective validation for the assessment of the upper gastrointestinal mucosal cleanliness during an esophagogastroduodenoscopy (EGD). The aim of this study was to develop a valid and reproducible cleanliness scale for use during an EGD. MethodsWe developed a cleanliness scale (Barcelona scale) with a score (0–2 points) of five segments of the upper gastrointestinal tract with thorough cleaning techniques (esophagus, fundus, body, antrum, and duodenum). First, 125 photos (25 of each area) were assessed, and a score was assigned to each image by consensus among 7 experts endoscopists. Subsequently, 100 of the 125 images were selected and the inter- and intra-observer variability of 15 previously trained endoscopists was evaluated using the same images at two different times. ResultsIn total, 1500 assessments were performed. In 1336/1500 observations (89%) there was agreement with the consensus score, with a mean kappa value of 0.83 (0.45–0.96). In the second evaluation, in 1330/1500 observations (89%) there was agreement with the consensus score, with a mean kappa value of 0.82 (0.45–0.93). The intra-observer variability was 0.89 (0.76–0.99). ConclusionsThe Barcelona cleanliness scale is a valid measure and reproducible with minimal training. Its application in clinical practice is a significant step to standardize the quality of the EGD.

The discrimination between autoimmune glial fibrillary acidic protein astrocytopathy and tuberculous meningitis

ObjectiveThe differential diagnosis between autoimmune glial fibrillary acidic protein astrocytopathy (AGFAPA) mimicking tuberculous meningitis and tuberculous meningitis (TBM) remains challenging in clinical practice. This study aims to identify the clinical, laboratory parameters, and clinical score systems that may be helpful in differentiating AGFAPA from TBM. MethodOverall 22 AGFAPA patients who were initially misdiagnosed as TBM (AGFAPA-TBM) and 30 confirmed TBM patients were included. The clinical, laboratory, imaging parameters, Thwaites systems, and Lancet consensus scoring systems (LCSS) of all patients were reviewed. Logistic regression was employed to establish a diagnostic formula to differentiate AGFAPA-TBM from TBM. The receiver operating characteristic (ROC) curve was applied to determine the best diagnostic critical point of the formula. ResultsUrinary retention was more frequent in AGFAPA-TBM patients (72.7% vs 33.3%, p = 0.012). A significantly lower ratio of T-SPOT. TB was noted in AGFAPA-TBM patients (9.1% vs 82.1%, p < 0.001). We found the LCSS was able to differentiate AGFAPA-TBM from TBM (AUC value 0.918, 95% CI=0.897–0.924). Furthermore, we set up a new scoring system with three variables: urinary retention, T-SPOT. TB, and cerebral imaging criteria in LCSS. The proposed diagnostic score ranges from -8 to 2, and a score of ≥ 0 was suggestive of AGFAPA-TBM (AUC value 0.938, 95% CI=0.878–0.951). ConclusionsThis study is the first to evaluate the Thwaites system and LCSS in AGFAPA-TBM and TBM. We provide an alternative diagnostic formula to differentiate AGFAPA-TBM from TBM and suggest testing for GFAP antibodies to avoid misdiagnosis when this scoring system meets AGFAPA-TBM.

Assessing the diagnostic utility of the Gaucher Earlier Diagnosis Consensus (GED-C) scoring system using real-world data

BackgroundGaucher disease (GD) is a rare autosomal recessive condition associated with clinical features such as splenomegaly, hepatomegaly, anemia, thrombocytopenia, and bone abnormalities. Three clinical forms of GD have been defined based on the absence (type 1, GD1) or presence (types 2 and 3) of neurological signs. Early diagnosis can reduce the likelihood of severe, often irreversible complications. The aim of this study was to validate the ability of factors from the Gaucher Earlier Diagnosis Consensus (GED-C) scoring system to discriminate between patients with GD1 and controls using real-world data from electronic patient medical records from Maccabi Healthcare Services, Israel’s second-largest state-mandated healthcare provider.MethodsWe applied the GED-C scoring system to 265 confirmed cases of GD and 3445 non-GD controls matched for year of birth, sex, and socioeconomic status identified from 1998 to 2022. The analyses were based on two databases: (1) all available data and (2) all data except free-text notes. Features from the GED-C scoring system applicable to GD1 were extracted for each individual. Patients and controls were compared for the proportion of the specific features and overall GED-C scores. Decision tree and random forest models were trained to identify the main features distinguishing GD from non-GD controls.ResultsThe GED-C scoring distinguished individuals with GD from controls using both databases. Decision tree models for the databases showed good accuracy (0.96 [95% CI 0.95–0.97] for Database 1; 0.95 [95% CI 0.94–0.96] for Database 2), high specificity (0.99 [95% CI 0.99–1]) for Database 1; 1.0 [95% CI 0.99–1] for Database 2), but relatively low sensitivity (0.53 [95% CI 0.46–0.59] for Database 1; 0.32 [95% CI 0.25–0.38]) for Database 2). The clinical features of splenomegaly, thrombocytopenia (< 50 × 109/L), and hyperferritinemia (300–1000 ng/mL) were found to be the three most accurate classifiers of GD in both databases.ConclusionIn this analysis of real-world patient data, certain individual features of the GED-C score discriminate more successfully between patients with GD and controls than the overall score. An enhanced diagnostic model may lead to earlier, reliable diagnoses of Gaucher disease, aiming to minimize the severe complications associated with this disease.

Comparison of S100A8 and PRAME as biomarkers fordistinguishing melanoma from melanocytic naevus: acase-control analysis.

S100A8 is a melanoma biomarker expressed in the melanoma-associated epidermal keratinocytes, but its diagnostic utility has not been compared with other biomarkers, including PRAME. To compare the utility of S100A8 and PRAME immunohistochemistry (IHC) in the differential diagnosis of melanoma and naevi in a case-control study. A previously described cohort of 209 melanomas (case samples) and naevi (control samples) dual-immunostained for S100A8 and PRAME were included. For S100A8, previously reported scores indicating the proportion of tumour-associated epidermis stained (0 = indeterminate; 1 = 0-4%; 2 = 5-25%; 3 = 26-50%; 4 = 51-75%; 5 = > 75%) were utilized. PRAME IHC was reviewed by at least two reviewers and a consensus score assigned, with score indicating the proportion of tumour stained (0 = indeterminate; 1 = 0%; 2 = 1-50%; 3 = > 50%). A positive test was defined as > 50% staining. The area under the receiver operating characteristic curves for S100A8 (0.833) and PRAME (0.874) were not significantly different from each other (P = 0.22). The diagnostic sensitivity and specificity were 42.4% [95% confidence interval (CI) 32.6-52.8%] and 98.2% (95% CI 93.6-99.8%) for S100A8, and 79.8% (95% CI 70.5-87.2%) and 87.3% (95% CI 79.6-92.9%) for PRAME, respectively. A combined test requiring both S100A8 and PRAME IHC positivity had a sensitivity of 39.4% (95% CI 29.7-49.7%) and specificity of 99.1% (95% CI 95.0-100.0%). S100A8 and PRAME have utility in the diagnostic workup of melanoma, with S100A8 being more specific and PRAME being more sensitive when using this threshold. Our findings suggest that these two immunohistochemical markers may favourably complement one another to improve the detection of melanoma.