Test Set Validation Research Articles

Development and validation of outcome prediction model for reperfusion therapy in acute ischemic stroke using nomogram and machine learning.

To develop logistic regression nomogram and machine learning (ML)-based models to predict 3-month unfavorable functional outcome for acute ischemic stroke (AIS) patients undergoing reperfusion therapy. Patients undergoing reperfusion therapy (intravenous thrombolysis and/or endovascular treatment) were prospectively recruited. Unfavorable outcome was defined as 3-month modified Rankin Scale (mRS) score 3-6. The independent risk factors associated with unfavorable outcome were obtained by regression analysis and included in the prediction model. The performance of nomogram was assessed by the area under the curve (AUC), calibration curve, and decision curve analysis (DCA). ML models were compared with nomogram using AUC; the generalizability of all models was ascertained in an external cohort. A total of 505 patients were enrolled, with 256 in the model construction, and 249 in the external validation. Five variables were identified as prognostic factors: baseline NIHSS, D-dimer level, random blood glucose (RBG), blood urea nitrogen (BUN), and systolic blood pressure (SBP) before reperfusion. The AUC values of nomogram were 0.865, 0.818, and 0.779 in the training set, test set, and external validation, respectively. The calibration curve and DCA indicated appreciable reliability and good net benefits. The best three ML models were extra trees (ET), CatBoost, and random forest (RF) models; all of them showed favorable discrimination in the training cohort, and confirmed in the test and external sets. Baseline NIHSS, D-dimer, RBG, BUN, and SBP before reperfusion were independent predictors for 3-month unfavorable outcome after reperfusion therapy in AIS patients. Both nomogram and ML models showed good discrimination and generalizability.

Development and validation of a multimodal model in predicting severe acute pancreatitis based on radiomics and deep learning

ObjectiveAim to establish a multimodal model for predicting severe acute pancreatitis (SAP) using machine learning (ML) and deep learning (DL). MethodsIn this multicentre retrospective study, patients diagnosed with acute pancreatitis at admission were enrolled from January 2017 to December 2021. Clinical information within 24 h and CT scans within 72 h of admission were collected. First, we trained Model α based on clinical features selected by least absolute shrinkage and selection operator analysis. Second, radiomics features were extracted from 3D-CT scans and Model β was developed on the features after dimensionality reduction using principal component analysis. Third, Model γ was trained on 2D-CT images. Lastly, a multimodal model, namely PrismSAP, was constructed based on aforementioned features in the training set. The predictive accuracy of PrismSAP was verified in the validation and internal test sets and further validated in the external test set. Model performance was evaluated using area under the curve (AUC), accuracy, sensitivity, specificity, recall, precision and F1-score. ResultsA total of 1,221 eligible patients were randomly split into a training set (n = 864), a validation set (n = 209) and an internal test set (n = 148). Data of 266 patients were for external testing. In the external test set, PrismSAP performed best with the highest AUC of 0.916 (0.873–0.960) among all models [Model α: 0.709 (0.618–0.800); Model β: 0.749 (0.675–0.824); Model γ: 0.687 (0.592–0.782); MCTSI: 0.778 (0.698–0.857); RANSON: 0.642 (0.559–0.725); BISAP: 0.751 (0.668–0.833); SABP: 0.710 (0.621–0.798)]. ConclusionThe proposed multimodal model outperformed any single-modality models and traditional scoring systems.

Detecting Avascular Necrosis of the Lunate from Radiographs Using a Deep-Learning Model

Deep-learning (DL) algorithms have the potential to change medical image classification and diagnostics in the coming decade. Delayed diagnosis and treatment of avascular necrosis (AVN) of the lunate may have a detrimental effect on patient hand function. The aim of this study was to use a segmentation-based DL model to diagnose AVN of the lunate from wrist postero-anterior radiographs. A total of 319 radiographs of the diseased lunate and 1228 control radiographs were gathered from Helsinki University Central Hospital database. Of these, 10% were separated to form a test set for model validation. MRI confirmed the absence of disease. In cases of AVN of the lunate, a hand surgeon at Helsinki University Hospital validated the accurate diagnosis using either MRI or radiography. For detection of AVN, the model had a sensitivity of 93.33% (95% confidence interval (CI) 77.93–99.18%), specificity of 93.28% (95% CI 87.18–97.05%), and accuracy of 93.28% (95% CI 87.99–96.73%). The area under the receiver operating characteristic curve was 0.94 (95% CI 0.88–0.99). Compared to three clinical experts, the DL model had better AUC than one clinical expert and only one expert had higher accuracy than the DL model. The results were otherwise similar between the model and clinical experts. Our DL model performed well and may be a future beneficial tool for screening of AVN of the lunate.

Building a Life Cycle Carbon Emission Estimation Model Based on an Early Design: 68 Case Studies from China

The building sector contributes to 50.9 percent of China’s carbon emissions. Due to the complexity of the assessment process, it is difficult to predict the entire life cycle carbon emissions of a building at the early stage of design. In this study, a whole-life carbon emission estimation model for the early stage of building design is developed based on comparison of the standard calculations and an analysis of stock cases. Firstly, the standard calculation methods in China, Japan and Europe were compared, and the boundary of the model was defined in three parts: production, construction and demolition and operation. Second, information on 68 examples of Chinese buildings was collected and divided into a training set and a test set at a ratio of 7:3. In the training set, the relationship between carbon emissions and the design parameters was searched, and a carbon emission estimation model applicable to different stages was constructed. Finally, the model was applied to the test set for validation. The results show that the calculation error of the model is within ±15%, and it can quickly estimate carbon emissions based on the design factors, which is helpful for carbon emission assessment work in the early stages of design.

MAGPIE: accurate pathogenic prediction for multiple variant types using machine learning approach

Identifying pathogenic variants from the vast majority of nucleotide variation remains a challenge. We present a method named Multimodal Annotation Generated Pathogenic Impact Evaluator (MAGPIE) that predicts the pathogenicity of multi-type variants. MAGPIE uses the ClinVar dataset for training and demonstrates superior performance in both the independent test set and multiple orthogonal validation datasets, accurately predicting variant pathogenicity. Notably, MAGPIE performs best in predicting the pathogenicity of rare variants and highly imbalanced datasets. Overall, results underline the robustness of MAGPIE as a valuable tool for predicting pathogenicity in various types of human genome variations. MAGPIE is available at https://github.com/shenlab-genomics/magpie.

Quantitative Predictive Studies of Multiple Biological Activities of TRPV1 Modulators.

TRPV1 channel agonists and antagonists, which have powerful analgesic effects without the addictive qualities associated with traditional analgesics, have become a focus area for the development of novel analgesics. In this study, quantitative structure-activity relationship (QSAR) models for three bioactive endpoints (Ki, IC50, and EC50) were successfully constructed using four machine learning algorithms: SVM, Bagging, GBDT, and XGBoost. These models were based on 2922 TRPV1 modulators and incorporated four types of molecular descriptors: Daylight, E-state, ECFP4, and MACCS. After the rigorous five-fold cross-validation and external test set validation, the optimal models for the three endpoints were obtained. For the Ki endpoint, the Bagging-ECFP4 model had a Q2 value of 0.778 and an R2 value of 0.780. For the IC50 endpoint, the XGBoost-ECFP4 model had a Q2 value of 0.806 and an R2 value of 0.784. For the EC50 endpoint, the SVM-Daylight model had a Q2 value of 0.784 and an R2 value of 0.809. These results demonstrate that the constructed models exhibit good predictive performance. In addition, based on the model feature importance analysis, the influence between substructure and biological activity was also explored, which can provide important theoretical guidance for the efficient virtual screening and structural optimization of novel TRPV1 analgesics. And subsequent studies on novel TRPV1 modulators will be based on the feature substructures of the three endpoints.

Development and Validation of a Machine Learning Model to Predict Post-dispatch Cancellation of Physician-staffed Rapid Car.

This study aimed to develop and validate a machine learning prediction model for post-dispatch cancellation of physician-staffed rapid car. Data were extracted from the physician-staffed rapid response car database at our Hospital between April 2017 and March 2019. After obtaining 2019 cases, we divided the dataset into a training set for developing the model and a test set for validation using stratified random sampling with an 8 : 2 allocation ratio. We selected random forest as the machine-learning classifier. The outcome was the post-dispatch cancellation of a rapid car. The model was trained using predictor variables, including 18 different reasons for rapid car request, age and gender of a patient, date (month), and distance from the hospital. This machine learning model predicted the occurrence of post-dispatch cancellation of rapid cars with an accuracy of 75.5% [95% confidence interval (CI): 71.0-79.6], sensitivity of 81.5% (CI: 75.0-86.9), specificity of 70.8% (CI: 64.4-76.6), and an area under the receiver operating characteristic value of 0.83 (CI: 0.79-0.87). The important features were distance from the hospital to the scene, age, suspicion of non-witnessed cardiac arrest, farthest geographic area, and date (months). We developed a favorable machine learning model to predict post-dispatch cancellation of rapid cars in a local district. This study suggests the potential of machine-learning models in improving the efficiency of dispatching physicians outside hospitals.

Algorithm Development and Early Performance Evaluation of a Next-Generation Multitarget Stool DNA Screening Test for Colorectal Cancer

Background and AimsThe multitarget stool DNA (mt-sDNA) assay is a non-invasive average-risk colorectal cancer (CRC) screening test. A new biomarker panel was developed for a next-generation test to improve specificity while maintaining/increasing sensitivity. We aimed first to establish an algorithm and cutoff for the next-generation mt-sDNA test and then to validate it using archived samples from the pivotal DeeP-C study (NCT01397747) of the first-generation test. MethodsAlgorithm development and cross-validation included 3011 samples from 2 specimen collection studies (NCT03821948, NCT03789162). The algorithm and cutoff were locked before validation. Validation test set samples included 57 CRC, 583 advanced precancerous lesions (APLs), and 7022 samples negative for CRC or APLs from the DeeP-C study, which prospectively enrolled average-risk, asymptomatic adults aged 50 to 84 years before screening colonoscopy. Next-generation biomarkers included methylated DNA markers LASS4, LRRC4, PPP2R5C, and ZDHHC1 (reference marker), and fecal hemoglobin. Primary validation endpoints were CRC sensitivity and specificity for the absence of advanced neoplasia. Secondary endpoints included APL sensitivity and specificity for non-neoplastic findings or negative colonoscopy. ResultsCross-validation and best-fit results from algorithm development closely matched, confirming algorithm reliability and reproducibility. For the test set, next-generation mt-sDNA test sensitivity was 93.0% (95% CI, 83.0%-98.1%) for CRC and 48.4% (95% CI, 44.2%-52.5%) for APLs. Specificity was 88.5% (95% CI, 87.7%-89.2%) for the absence of advanced neoplasia and 90.4% (95% CI, 89.5%-91.2%) for the combination of non-neoplastic findings or negative colonoscopy. ConclusionsBased on archived samples, the next-generation mt-sDNA test demonstrated promising CRC screening performance characteristics that will be further assessed in a prospective clinical validation study (BLUE-C; NCT04144738).

Design of some potent non-toxic Autoimmune disorder inhibitors based on 2D-QSAR, CoMFA, molecular docking, and molecular dynamics investigations

Current clinical research suggests that inhibitors of protein arginine deiminase 4 (PAD4), major histocompatibility complex (MHC) class II HLA-DQ-ALPHA chain, and thyrotropin receptor (or TSH receptor) which are of biological and therapeutic interest, may show potential in treating rheumatoid arthritis, type 1 diabetes, Graves' disease and other autoimmune disorder. In the present study, a comprehensive analysis was conducted on a collection of 32 compounds concerning their anti-rheumatoid arthritis activity as inhibitors of PAD4. This analysis represents the first instance in which these compounds were computationally examined, employing an in-silico approach that considered 2D-3D QSAR modeling, and molecular docking and was further validated through molecular dynamics and ADMET properties assessment. A credible 2D QSAR (Q_LOO^2 ​= ​0.6611 and R^2 ​= ​0.7535) model was constructed and verified using an external validation test set, Y-randomization, variance inflation factor (VIF), mean effect (MF), and William's plot applicability domain (AD). Ligand-based alignment was implemented in the 3D-QSAR examination. The outcomes demonstrated that CoMFA (uvepls) (Q2LOO ​= ​0.5877; R2 ​= ​0.9983) possess remarkable stability and foresight. The internal validation indicated that CoMFA (uvepls) MIFs display superior predictive capability compared to COMFA (ffdsel). Structural criteria determined by the contour maps of the model and molecular docking simulations were strategically employed to computationally develop 10 new, non-toxic autoimmune disease inhibitors with increased efficacy. Docking tests were done on the newly developed compounds to illustrate their binding mechanism and to identify critical interaction residues inside the active region of rheumatoid arthritis (PDB id: 3BLU). In addition, docking results of the selected designed compounds inside the active sites of type 1 diabetes receptor (6DFX), and Graves' disease receptor (4QT5) demonstrated their rheumatoid arthritis (PDB id: 3BLU) selectivity. A molecular dynamics simulation and binding free energy calculations using the MM/GBSA technique confirmed the stability of the proposed compound D4 inside the rheumatoid arthritis (3BLU) receptor active site. In summary, the results of our investigation might give considerable insight into the future design and development of new autoimmune disease inhibitors.

2-[18F]FDG PET-based quantification of lymph node metabolic heterogeneity for predicting lymph node metastasis in patients with colorectal cancer.

The pre-surgical estimation of lymph node (LN) metastasis in colorectal cancer (CRC) poses a significant diagnostic predicament. The associations between LN morphology, density, and metabolic heterogeneity and LN metastasis status in CRCs have been seldomly examined through the lens of radiomics. This research aimed to assess 2-[18F]FDG PET-based quantification of intratumoral metabolic heterogeneity for predicting lymph node metastasis in patients with colorectal cancer. The construction of the model utilized data from 264 CRC patients, all of whom underwent preoperative 2-[18F]FDG PET/CT. Radiomic features were extracted from PET and CT images of LNs. Least absolute shrinkage and selection operator (LASSO) regression was implemented for selecting pertinent imaging features with a tenfold cross-validation. The predictive accuracy for LN metastasis status was juxtaposed against traditional methodologies (comprising CT-reported LN status and PET/CT-reported LN status) by deploying the receiver operating characteristic (ROC) curve analysis. The radiomics signature was evaluated based on discrimination, calibration, and clinical utility parameters. The model was further subjected to validation using an independent cohort of 132 patients from the period of January 2012 to June 2020. The radiomics model was composed of eight significant radiomic features (five from PET and three from CT), encapsulating metabolic and density heterogeneity. The radiomics signature (area under the curve (AUC), 0.908) showcased a significantly superior performance compared to CT-reported LN status (AUC, 0.563, P < 0.001) and PET/CT-reported LN status (AUC, 0.64, P < 0.001) for predicting LN-positive or LN-negative status. The radiomics signature (AUC, 0.885) also showcased a significantly superior performance compared to CT-reported LN status (AUC, 0.587, P < 0.001) and PET/CT-reported LN status (AUC, 0.621, P < 0.001) to identify N1 and N2. This signature maintained its independence from clinical risk factors and exhibited robustness in the validation test set. Decision curve analysis attested to the clinical utility of the radiomics signature. The radiomics signature based on 2-[18F]FDG PET/CT, which derived image features directly from LNs irrespective of clinical risk factors, displayed enhanced diagnostic performance compared to conventional CT or PET/CT-reported LN status. This allows for the identification of pre-surgical LN metastasis status and facilitates a patient-specific prediction of LN metastasis status in CRC patients.

The potential of near–infrared spectroscopy as a rapid method for quality evaluation of cassava leaves and roots

In this study, near–infrared spectroscopy (NIRS) was used to predict quality of cassava leaves and roots based on their own spectra and also to predict root quality based on the spectral data of the leaves. 990 cassava leaves and 330 roots were collected from 110 plants. The calibration and validation models were constructed by partial least square models with test set validation. High prediction performance models were found for predicting leaf dry matter (DM) and total cyanide (HCN) content using leaf spectra. The coefficient of determination of the validation set (R2val), root means square error of prediction (RMSEP), and the ratio performance deviation (RPDval) of leaf DM were 0.891, 1.04%, and 3.03, respectively, and those of leaf HCN were 0.909, 60.5 ppm, and 3.32, respectively. The prediction model developed from root spectra showed an excellent prediction result for root DM (R2val = 0.979, RMSEP = 0.677%, RPDval = 6.92), while good predicting ability models were obtained for HCN, starch, and TSS in root. Moreover, prediction models developed from leaf spectra could be used for qualitative screening of root quality. This study showed the potential of using NIRS as a rapid and reliable method to predict cassava quality at harvest or before processing.

Integrating machine learning and high throughput screening for the discovery of allosteric AKT1 inhibitors

Evidence from clinical and experimental investigations reveals the role of AKT in oral cancer, which has led to the development of therapeutic and pharmacological medications for inhibiting AKT protein. Despite prodigious effort, researchers are searching for new allosteric inhibitors as orthosteric inhibitors are non-selective and exert off-target effects. In the current study, we proposed an integrated computational workflow for identifying allosteric AKT1 inhibitors as this isoform is highly correlated with poor prognosis and survival. To achieve this objective, 84 classification QSAR models with six different machine learning algorithms were developed. The models created with RDKit_RF and RDKit_kstar outperformed internal and test set validation with an ROC of 0.98. The outperformed models were then used to screen Chembl, which contains over a million compounds, for AKT1 inhibitors. The Tanimoto similarity search approach identified the compounds structurally resembling AKT allosteric inhibitors. The filtered compounds were further subjected to docking phases, molecular dynamic simulation and mmpbsa to verify the binding mode of selected ones. All these analyses suggested hit 5 (CHEMBL3948083) as the potential allosteric inhibitor of AKT1 as the stability parameters, favourable binding affinity (-107.78 ± 11.56 KJ/mol) and ligand interaction were better in comparison to other compounds and reference compound. The residual analysis demonstrated that allosteric and isoform-specific residues such as Trp80 and Val270 contributed the larger energy for ligand binding. The proposed integrated approach in this study might achieve a futuristic outcome when employed in a pharmaceutical scheme different from the conventional method. Communicated by Ramaswamy H. Sarma

Abstract B016: AI analysis of histological images accurately identifies luminal subtype urothelial carcinomas characterized by high PPARG expression

Abstract Background PPARG is a cell lineage determining transcription factor in muscle invasive urothelial carcinoma (MIUC), where high expression is associated with the luminal subtype (1, 2). As FX-909(2), a PPARG inverse agonist, enters the clinic, biomarkers that reflect the luminal subtype will reveal patients with the potential to respond to PPARG inhibition. The determination of luminal status is generally performed via RNA-seq and/or multiple immunohistochemistry stains, which are costly and time-consuming. However, MIUC biopsies are routinely stained with hematoxylin and eosin (H&amp;E). Machine learning (ML)-driven analyses of H&amp;E-stained tissue may enable the identification of patients with luminal MIUC and have advantages over the current molecular approach. Methods H&amp;E-stained slides from 367 unique primary MIUC cases from the TCGA BLCA dataset were split into training (70%), validation (15%), and held-out test (15%) sets by preserving the data distribution of patient metadata. A curated retrospective cohort of 42 localized, stage III-IV primary MIUCs was used as an independent test set. Molecular classification as luminal (luminal papillary, luminal, and luminal infiltrated subtypes) or non-luminal (basal-squamous and neuronal subtypes) was performed and used as ground truth(3). Pretrained artifact and tissue segmentation models were deployed on all images to identify artifact-free areas of cancer and cancer-associated stroma. An end-to-end (E2E) additive multiple instance learning model was trained to identify luminal cases using the training set. Top performing model iterations were compared on the validation set, and the optimal iteration was deployed on both test sets. Results We assessed the performance of our E2E model in predicting luminal status using the molecular subtypes derived from Robertson et al. as ground truth(3). The E2E model showed excellent performance when predicting luminal status in the TCGA validation, TCGA test, and independent test sets (AUROC = 0.96, 0.95, and 0.97, respectively). The accuracy in all three cohorts was 89-90%, with a sensitivity of 0.86-0.96, a specificity of 0.82-0.94, and an F1 score of 0.88-0.9. Conclusions We generated a robust ML model that accurately predicts luminal MIUC using H&amp;E-stained slides. Luminal MIUC is dependent on PPARG, and PPARG inverse agonism represents a promising therapeutic approach for MIUC. Coupled with the first-in-class FX-909 therapeutic entering the clinic, the strong performance of our model highlights the potential for its application as a precision biomarker to identify patients with advanced urothelial carcinoma likely to respond to PPARG inhibition.

Development and validation of a natural language processing system that extracts cognitive test results from clinical notes

AbstractBackgroundCognitive test results from electronic health records (EHRs) are key information for assessing the severity and progression of patients with mild cognitive impairment (MCI) and Alzheimer’s’ disease (AD). However, such information is often recorded in unstructured clinical notes rather than in an administrative database. We developed and validated a natural language processing (NLP) system to extract cognitive test results from clinical notes in the Veterans Affair (VA) Healthcare System.MethodAn NLP system was developed using regular expression‐based rules and Python to extract results for six tests that have been used most frequently in VA: Mini‐Mental State Exam (MMSE), Montreal Cognitive Assessment (MoCA), Saint Louis University Mental Status Examination (SLUMS), Mini‐cog, Boston Naming Test (BNT), and Benton Visual Retention Test (BVRT). The system extracted test results from each note in two steps: (1) searched a test name, or variations and abbreviation of the test name and, if successful, (2) searched the quantitative results (e.g., 12/30 for MMSE, 74th percentile for BNT) and/or descriptive results (e.g., “borderline impairment” for BVRT) within 5 words or one sentence before or after the test name. To balance the system performance and speed, we developed 3‐8 extraction rules per test based on a manual review of 30‐50 notes for each test. We further validated NLP performance on 6 held‐out datasets (50‐200 notes/test). We automatically sampled the development and held‐out notes by searching the test name and its variations/abbreviation to increase positive cases, i.e., notes that contained the above test results.ResultThe NLP system achieved 0.72‐0.92 predictive positive values (PPV), 0.96‐1.00 recall, and 0.83‐0.95 F1 scores on the validation test sets (Table 1). In addition, it demonstrated a scalable performance (processed 200,000 notes in 7 min), allowing an extraction of millions of notes from ∼1 million patients within hours.ConclusionRule‐based NLP can extract cognitive test results with adequate performance and scalable capability for clinical notes from patients with MCI or AD within the VA Healthcare System.

Deep learning models for automatic tumor segmentation and total tumor volume assessment in patients with colorectal liver metastases

BackgroundWe developed models for tumor segmentation to automate the assessment of total tumor volume (TTV) in patients with colorectal liver metastases (CRLM).MethodsIn this prospective cohort study, pre- and post-systemic treatment computed tomography (CT) scans of 259 patients with initially unresectable CRLM of the CAIRO5 trial (NCT02162563) were included. In total, 595 CT scans comprising 8,959 CRLM were divided into training (73%), validation (6.5%), and test sets (21%). Deep learning models were trained with ground truth segmentations of the liver and CRLM. TTV was calculated based on the CRLM segmentations. An external validation cohort was included, comprising 72 preoperative CT scans of patients with 112 resectable CRLM. Image segmentation evaluation metrics and intraclass correlation coefficient (ICC) were calculated.ResultsIn the test set (122 CT scans), the autosegmentation models showed a global Dice similarity coefficient (DSC) of 0.96 (liver) and 0.86 (CRLM). The corresponding median per-case DSC was 0.96 (interquartile range [IQR] 0.95–0.96) and 0.80 (IQR 0.67–0.87). For tumor segmentation, the intersection-over-union, precision, and recall were 0.75, 0.89, and 0.84, respectively. An excellent agreement was observed between the reference and automatically computed TTV for the test set (ICC 0.98) and external validation cohort (ICC 0.98). In the external validation, the global DSC was 0.82 and the median per-case DSC was 0.60 (IQR 0.29–0.76) for tumor segmentation.ConclusionsDeep learning autosegmentation models were able to segment the liver and CRLM automatically and accurately in patients with initially unresectable CRLM, enabling automatic TTV assessment in such patients.Relevance statementAutomatic segmentation enables the assessment of total tumor volume in patients with colorectal liver metastases, with a high potential of decreasing radiologist’s workload and increasing accuracy and consistency.Key points• Tumor response evaluation is time-consuming, manually performed, and ignores total tumor volume.• Automatic models can accurately segment tumors in patients with colorectal liver metastases.• Total tumor volume can be accurately calculated based on automatic segmentations.Graphical

A fully automated pipeline for the extraction of pectoralis muscle area from chest computed tomography scans.

Computed tomography (CT)-derived pectoralis muscle area (PMA) measurements are prognostic in people with or at-risk of COPD, but fully automated PMA extraction has yet to be developed. Our objective was to develop and validate a PMA extraction pipeline that can automatically: 1) identify the aortic arch slice; and 2) perform pectoralis segmentation at that slice. CT images from the Canadian Cohort of Obstructive Lung Disease (CanCOLD) study were used for pipeline development. Aorta atlases were used to automatically identify the slice containing the aortic arch by group-based registration. A deep learning model was trained to segment the PMA. The pipeline was evaluated in comparison to manual segmentation. An external dataset was used to evaluate generalisability. Model performance was assessed using the Dice-Sorensen coefficient (DSC) and PMA error. In total 90 participants were used for training (age 67.0±9.9 years; forced expiratory volume in 1 s (FEV1) 93±21% predicted; FEV1/forced vital capacity (FVC) 0.69±0.10; 47 men), and 32 for external testing (age 68.6±7.4 years; FEV1 65±17% predicted; FEV1/FVC 0.50±0.09; 16 men). Compared with manual segmentation, the deep learning model achieved a DSC of 0.94±0.02, 0.94±0.01 and 0.90±0.04 on the true aortic arch slice in the train, validation and external test sets, respectively. Automated aortic arch slice detection obtained distance errors of 1.2±1.3 mm and 1.6±1.5 mm on the train and test data, respectively. Fully automated PMA measurements were not different from manual segmentation (p>0.05). PMA measurements were different between people with and without COPD (p=0.01) and correlated with FEV1 % predicted (p<0.05). A fully automated CT PMA extraction pipeline was developed and validated for use in research and clinical practice.

Toward AI-Assisted Clinical Assessment for Patients with Multiple Myeloma: Feature Selection for Large Language Models

Large language models (LLM) can potentially revolutionize the healthcare industry. They could reduce the burden on healthcare, increase care accessibility in areas with shortage and provide multilingual support to break down language barriers. Although, these models equipped with vast amounts of medical knowledge and the ability to understand and generate human-like text, require a proper set of feeding data (i.e., prompt engineering) for accurate diagnosis and providing reliable personalized treatment plans for patients. Multiple Myeloma (MM) is a complex hematological malignancy characterized by the uncontrolled proliferation of plasma cells in the bone marrow. Disease management for MM becomes particularly challenging due to its multisystemic nature based on the varying volume of malignant cells within the bone marrow. Implementing LLMs for clinical assessment of patients with MM needs feature selection to develop the most effective prompt for these models. Here, we utilized a machine learning (ML) approach to define salient features in a typical visit day that correlates most significantly with disease volume on the same day. These features could be the best candidate to reflect the multisystemic and dynamic nature of MM in each visit, and they could be candidates to be incorporated into LLMs to develop a system-based assessment in clinic visits. Methods: This study examined 1,472 clinical observations. To select a curated list of features associated with same-day M-spike values, 43 clinical and lab variables were input into an ML model. Random Forest (RF), an ensemble of regression trees suitable for nonlinear multiple regression, was selected as the model. The data were randomly divided into a training set (80%) and a test set (20%) for model validation. Using bootstrapping and generating 500 data sets, a random forest of regression trees was constructed, and results and estimates were aggregated across the trees. To determine the importance of each covariate, their inclusion and exclusion were compared in the models. Results: The residual distribution of the RF model indicated that nearly all M-spike values determined using the 43 variables distributed equally on either side of zero (Fig. 1). The weighted value of each of the 43 independent variables was determined by individually removing a variable from the ML algorithm and measuring its effect on the mean squared error (MSE) (Fig.2). Removal of the first lagged M-spike, serum total protein, second-lagged M-spike, serum IgG, serum IgM, and serum IgA, had the greatest effects on the ML algorithm. M-spike values determined using the ML algorithm correlated highly with M-spike values determined using the laboratory measured SPEP values as indicated by the proximity of the Pearson and Spearman correlation coefficients to +1. Using the 43 variables, the Pearson coefficient was 0.96 and the Spearman coefficient was 0.91. Feature selected modeling was performed to reduce the variables needed to predict the M-spike. Five RF models with different predictors were selected for comparison. Model A included all 43 predictors, Model B included the ten most important variables, Model C the top five variables, Model D included the first and second-lagged M-spike and serum total protein, and Model E the first-lagged M-spike and serum total protein. The Pearson's r and RMSE (root mean square error) values were used to compare the models. Pearson's r values for Model A, B, C, and D were 0.96, 0.96, 0.96, and 0.95 respectively, and the RMSE values were 0.21, 0.19, 0.19, and 0.22. In Model E, feature selection used only two variables and accurately predicted the M-spike value (Pearson's r = 0.95; RMSE 0.22). The Pearson's r values for feature selected models A, B, C, D, and E were 0.95, 0.96, 0.96, 0.95 and 0.91. Conclusion: Accurate prompt engineering to create global assessment of Myeloma clone by LLM requires a curated set of variables that correlated with disease volume. Here, we developed an ML model for feature selection utilizing the same-day available data in the patient chart. Features could be used in order of importance to provide focused, comprehensive prompts that aligned with the patient's context. The quality of AI-assisted disease assessment using these models should be compared with assessments performed by real world providers in future studies to ensure that the LLMs generate a written assessment that accurately reflects the patient's health status.

Distinguishing infectivity in patients with pulmonary tuberculosis using deep learning.

This study aimed to develop and assess a deep-learning model based on CT images for distinguishing infectivity in patients with pulmonary tuberculosis (PTB). We labeled all 925 patients from four centers with weak and strong infectivity based on multiple sputum smears within a month for our deep-learning model named TBINet's training. We compared TBINet's performance in identifying infectious patients to that of the conventional 3D ResNet model. For model explainability, we used gradient-weighted class activation mapping (Grad-CAM) technology to identify the site of lesion activation in the CT images. The TBINet model demonstrated superior performance with an area under the curve (AUC) of 0.819 and 0.753 on the validation and external test sets, respectively, compared to existing deep learning methods. Furthermore, using Grad-CAM, we observed that CT images with higher levels of consolidation, voids, upper lobe involvement, and enlarged lymph nodes were more likely to come from patients with highly infectious forms of PTB. Our study proves the feasibility of using CT images to identify the infectivity of PTB patients based on the deep learning method.

A radiomics based approach using adrenal gland and periadrenal fat CT images to allocate COVID-19 health care resources fairly

BackgroundThe value of radiomics features from the adrenal gland and periadrenal fat CT images for predicting disease progression in patients with COVID-19 has not been studied extensively. We assess the value of radiomics features from the adrenal gland and periadrenal fat CT images in predicting COVID-19 disease exacerbation.MethodsA total of 1,245 patients (685 moderate and 560 severe patients) were enrolled in a retrospective study. We proposed a 3D V-net to segment adrenal glands in onset CT images automatically, and periadrenal fat was obtained using inflation operation around the adrenal gland. Next, we built a clinical model (CM), three radiomics models (adrenal gland model [AM], periadrenal fat model [PM], and fusion of adrenal gland and periadrenal fat model [FM]), and radiomics nomogram (RN) after radiomics features extracted.ResultsThe auto-segmentation framework yielded a dice value 0.79 in the training set. CM, AM, PM, FM, and RN obtained AUCs of 0.717, 0.716, 0.736, 0.760, and 0.833 in the validation set. FM and RN had better predictive efficacy than CM (P < 0.0001) in the training set. RN showed that there was no significant difference in the validation set (mean absolute error [MAE] = 0.04) and test set (MAE = 0.075) between predictive and actual results. Decision curve analysis showed that if the threshold probability was between 0.4 and 0.8 in the validation set or between 0.3 and 0.7 in the test set, it could gain more net benefits using RN than FM and CM.ConclusionsRadiomics features extracted from the adrenal gland and periadrenal fat CT images are related to disease exacerbation in patients with COVID-19.

A novel deep learning model for a coronary computed tomography angiography diagnosis of plaque erosion

Abstract Background Plaque erosion is the underlying pathology in up to 50% of patients with non-ST-segment elevation acute coronary syndromes (ACS). Previous studies suggest that patients with erosion might be treated conservatively without stenting. Currently, a diagnosis of plaque erosion requires invasive intracoronary imaging. Purpose We sought to develop a deep learning (DL) model that enables an accurate diagnosis of plaque erosion using coronary computed tomography angiography (CTA). Methods A total of 532 coronary CTA scans (143 coronary CTA scans with plaque rupture, 113 with plaque erosion, and 276 with reference segment) from 395 patients were included to develop a model: 426 coronary CTA scans from 316 patients for training and internal validation, and 106 separate scans from 79 patients for test set validation. Among ACS patients, the diagnosis of plaque rupture or erosion identified by OCT was used as the ground truth, and the corresponding site on the CTA image matched with OCT image was determined to be the culprit lesion. A novel DL, Momentum Distillation Composite Transformer Attention (MD-CTA) model, was developed that can effectively process the whole CTA scans to diagnose plaque erosion. Results The novel DL model showed diagnostic performance in the five-fold cross-validation with an area under the receiver-operating characteristic curve (AUC) of 0.901 (0.873-0.930), a sensitivity of 81.2 (72.8-88.0), and a specificity of 86.6 (82.4-90.2), all of which were significantly higher than those of convolutional neural network (CNN) model with an AUC of 0.621 (0.567-0.675), the sensitivity of 59.8 (50.1-69.0), and specificity of 60.2 (54.5-65.7) in training validation. Similarly, in the test set validation, the AUC, sensitivity, and specificity of the MD-CTA model were 0.899 (0.841-0.957), 87.1 (70.2-96.4), and 85.3 (75.3-92.4), respectively, higher than those of 0.724 (0.622-0.826), 71.0 (52.0-85.8), and 68.0 (56.2-78.3) of the CNN model. At the slice level, the MD-CTA model was also superior in the five-fold cross-validation and the test set validation compared to the CCN model. This model significantly outperformed experienced cardiologists, especially for sensitivity. Conclusions The newly developed DL model enables an accurate CTA diagnosis of plaque erosion, which might enable the cardiologist to provide tailored therapy without invasive procedures. This algorithm might lead to a major shift in managing millions of patients with plaque erosion each year.Table 1Figure 1