Multi-label Classification Model Research Articles

Deep active learning for multi label text classification

Given a set of labels, multi-label text classification (MLTC) aims to assign multiple relevant labels for a text. Recently, deep learning models get inspiring results in MLTC. Training a high-quality deep MLTC model typically demands large-scale labeled data. And comparing with annotations for single-label data samples, annotations for multi-label samples are typically more time-consuming and expensive. Active learning can enable a classification model to achieve optimal prediction performance using fewer labeled samples. Although active learning has been considered for deep learning models, there are few studies on active learning for deep multi-label classification models. In this work, for the deep MLTC model, we propose a deep Active Learning method based on Bayesian deep learning and Expected confidence (BEAL). It adopts Bayesian deep learning to derive the deep model’s posterior predictive distribution and defines a new expected confidence-based acquisition function to select uncertain samples for deep MLTC model training. Moreover, we perform experiments with a BERT-based MLTC model, where BERT can achieve satisfactory performance by fine-tuning in various classification tasks. The results on benchmark datasets demonstrate that BEAL enables more efficient model training, allowing the deep model to achieve training convergence with fewer labeled samples.

Hierarchical multi-label classification model for science and technology news based on heterogeneous graph semantic enhancement

In the context of high-quality economic development, technological innovation has emerged as a fundamental driver of socio-economic progress. The consequent proliferation of science and technology news, which acts as a vital medium for disseminating technological advancements and policy changes, has attracted considerable attention from technology management agencies and innovation organizations. Nevertheless, online science and technology news has historically exhibited characteristics such as limited scale, disorderliness, and multi-dimensionality, which is extremely inconvenient for users of deep application. While single-label classification techniques can effectively categorize textual information, they face challenges in leading science and technology news classification due to a lack of a hierarchical knowledge framework and insufficient capacity to reveal knowledge integration features. This study proposes a hierarchical multi-label classification model for science and technology news, enhanced by heterogeneous graph semantics. The model captures multi-dimensional themes and hierarchical structural features within science and technology news through a hierarchical transmission module. It integrates graph convolutional networks to extract node information and hierarchical relationships from heterogeneous graphs, while also incorporating prior knowledge from domain knowledge graphs to address data scarcity. This approach enhances the understanding and classification capabilities of the semantics of science and technology news. Experimental results demonstrate that the model achieves precision, recall, and F1 scores of 84.21%, 88.89%, and 86.49%, respectively, significantly surpassing baseline models. This research presents an innovative solution for hierarchical multi-label classification tasks, demonstrating significant application potential in addressing data scarcity and complex thematic classification challenges.

Multi-level information fusion for missing multi-label learning based on stochastic concept clustering

Missing multi-label learning is to address the problem of missing labels in multi-label datasets for multi-label classification tasks. Notably, the complex dependencies that typically exist between labels make accurate classification particularly challenging in the presence of missing labels. Some existing missing multi-label classification models often utilize feature selection to effectively recognize the dependencies between labels and features. However, they are ineffective at capturing hierarchical relationships of feature information, probably leading to a decline in prediction performance. To address this problem, this paper proposes a missing multi-label classification model based on multi-level stochastic concept clustering (MML-MSCC) to make dependencies between features and labels recognized more accurately and prediction performance better. In our model, optimal granularity selection is achieved through the global mutual information between features and labels, which makes the study of stochastic granule concept across multiple granularities. Furthermore, we utilize a stochastic concept clustering method to combine similar feature information for the purpose of making the missing label completion more reasonable. Note that stochastic granule concept clustering is performed with cross-granularity, thereby effectively capturing hierarchical relationships among feature information. Finally, to evaluate the performance of our model, we compare the MML-MSCC model with 9 existing missing multi-label classification models on 12 open datasets in terms of six evaluation metrics.

A Multi-label Artificial Intelligence Approach for Improving Breast Cancer Detection With Mammographic Image Analysis.

Breast cancer remains a major global health concern. This study aimed to develop a deep-learning-based artificial intelligence (AI) model that predicts the malignancy of mammographic lesions and reduces unnecessary biopsies in patients with breast cancer. In this retrospective study, we used deep-learning-based AI to predict whether lesions in mammographic images are malignant. The AI model learned the malignancy as well as margins and shapes of mass lesions through multi-label training, similar to the diagnostic process of a radiologist. We used the Curated Breast Imaging Subset of Digital Database for Screening Mammography. This dataset includes annotations for mass lesions, and we developed an algorithm to determine the exact location of the lesions for accurate classification. A multi-label classification approach enabled the model to recognize malignancy and lesion attributes. Our multi-label classification model, trained on both lesion shape and margin, demonstrated superior performance compared with models trained solely on malignancy. Gradient-weighted class activation mapping analysis revealed that by considering the margin and shape, the model assigned higher importance to border areas and analyzed pixels more uniformly when classifying malignant lesions. This approach improved diagnostic accuracy, particularly in challenging cases, such as American College of Radiology Breast Imaging-Reporting and Data System categories 3 and 4, where the breast density exceeded 50%. This study highlights the potential of AI in improving the diagnosis of breast cancer. By integrating advanced techniques and modern neural network designs, we developed an AI model with enhanced accuracy for mammographic image analysis.

Graph neural network-based subgraph analysis for predicting adverse drug events

PurposeAdverse drug events (ADEs) are a significant global public health concern, and they have resulted in high rates of hospital admissions, morbidity, and mortality. Prior to the use of machine learning and deep learning methods, ADEs may not become well recognized until long after a drug has been approved and is widely used, which poses a significant challenge for ensuring patient safety. Consequently, there is a need to develop computational approaches for earlier identification of ADEs not detected during pre-registration clinical trials. MethodsThis paper presents a state-of-the-art network-based approach that models patients as subgraphs composed of nodes of International Classification of Diseases (ICD) codes and directed edges illustrating disease progression. Four Graph Neural Network (GNN) variants were employed to make sub-graph level predictions that answer three Research Questions (RQ): 1) whether ADE(s) would occur given a patient's prior diagnoses history, 2) when an ADE would occur, and 3) which ADE would occur. The first and second RQs were addressed using a binary classification approach. The third RQ was addressed using a multi-label classification model. ResultsThe proposed network-based approach demonstrated superior performance in predicting ADEs, with the GraphSage model exhibiting the highest accuracy for both RQ 1 (0.8863) and RQ 3 (0.9367), while the Graph Attention Networks (GAT) model was found to perform best for RQ 2 (0.8769). Furthermore, an analysis segmented by ADE classification revealed that while RQs 1 and 3 exhibited minimal variance across different ADE categories, a distinct advantage was observed for categories B, C, and E in the context of RQ 2 when applying this sub-graph method. ConclusionThe network-based approach demonstrates the potential of GNNs in supporting the early detection and prevention of ADEs. Accurately predicting ADEs could enable healthcare professionals to make informed clinical decisions, take preventive measures and adjust medication regimens before serious adverse events occur. The proposed prediction method could also lead to optimized usage of healthcare resources by preventing hospital admissions and reducing the overall burden of adverse drug events on the healthcare systems.

Deep-learning model accurately classifies multi-label lung ultrasound findings, enhancing diagnostic accuracy and inter-reader agreement.

Despite the increasing use of lung ultrasound (LUS) in the evaluation of respiratory disease, operators' competence constrains its effectiveness. We developed a deep-learning (DL) model for multi-label classification using LUS and validated its performance and efficacy on inter-reader variability. We retrospectively collected LUS and labeled as normal, B-line, consolidation, and effusion from patients undergoing thoracentesis at a tertiary institution between January 2018 and January 2022. The development and internal testing involved 7580 images fromJanuary 2018and December2020, andthe model's performance was validated on a temporally separated test set (n = 985 imagescollected after January 2021) and two external test sets (n = 319 and 54 images). Two radiologists interpreted LUS with and without DL assistance and compared diagnostic performance and agreement. The model demonstrated robust performancewith AUCs: 0.93 (95% CI 0.92-0.94) for normal, 0.87 (95% CI 0.84-0.89) for B-line, 0.82 (95% CI 0.78-0.86) for consolidation, and 0.94 (95% CI 0.93-0.95) for effusion.The model improved reader accuracy for binary discrimination (normal vs. abnormal; reader 1: 87.5-95.6%, p = 0.004;reader 2: 95.0-97.5%, p = 0.19), and agreement(k = 0.73-0.83, p = 0.01). In conclusion,the DL-based model may assist interpretation, improving accuracy and overcoming operator competence limitationsin LUS.

Dual-stream multi-label image classification model enhanced by feature reconstruction

Dual-stream multi-label image classification model enhanced by feature reconstruction

Advanced Multi-Label Fire Scene Image Classification via BiFormer, Domain-Adversarial Network and GCN

Detecting wildfires presents significant challenges due to the presence of various potential targets in fire imagery, such as smoke, vehicles, and people. To address these challenges, we propose a novel multi-label classification model based on BiFormer’s feature extraction method, which constructs sparse region-indexing relations and performs feature extraction only in key regions, thereby facilitating more effective capture of flame characteristics. Additionally, we introduce a feature screening method based on a domain-adversarial neural network (DANN) to minimize misclassification by accurately determining feature domains. Furthermore, a feature discrimination method utilizing a Graph Convolutional Network (GCN) is proposed, enabling the model to capture label correlations more effectively and improve performance by constructing a label correlation matrix. This model enhances cross-domain generalization capability and improves recognition performance in fire scenarios. In the experimental phase, we developed a comprehensive dataset by integrating multiple fire-related public datasets, and conducted detailed comparison and ablation experiments. Results from the tenfold cross-validation demonstrate that the proposed model significantly improves recognition of multi-labeled images in fire scenarios. Compared with the baseline model, the mAP increased by 4.426%, CP by 4.14% and CF1 by 7.04%.

Clustered Automated Machine Learning (CAML) model for clinical coding multi-label classification

AbstractClinical coding is a time-consuming task that involves manually identifying and classifying patients’ diseases. This task becomes even more challenging when classifying across multiple diagnoses and performing multi-label classification. Automated Machine Learning (AutoML) techniques can improve this classification process. However, no previous study has developed an AutoML-based approach for multi-label clinical coding. To address this gap, a novel approach, called Clustered Automated Machine Learning (CAML), is introduced in this paper. CAML utilizes the AutoML library Auto-Sklearn and cTAKES feature extraction method. CAML clusters binary diagnosis labels using Hamming distance and employs the AutoML library to select the best algorithm for each cluster. The effectiveness of CAML is evaluated by comparing its performance with that of the Auto-Sklearn model on five different datasets from the Medical Information Mart for Intensive Care (MIMIC III) database of reports. These datasets vary in size, label set, and related diseases. The results demonstrate that CAML outperforms Auto-Sklearn in terms of Micro F1-score and Weighted F1-score, with an overall improvement ratio of 35.15% and 40.56%, respectively. The CAML approach offers the potential to improve healthcare quality by facilitating more accurate diagnoses and treatment decisions, ultimately enhancing patient outcomes.

Multi-Source Image Fusion Based Regional Classification Method for Apple Diseases and Pests

Efficient diagnosis of apple diseases and pests is crucial to the healthy development of the apple industry. However, the existing single-source image-based classification methods have limitations due to the constraints of single-source input image information, resulting in low classification accuracy and poor stability. Therefore, a classification method for apple disease and pest areas based on multi-source image fusion is proposed in this paper. Firstly, RGB images and multispectral images are obtained using drones to construct an apple diseases and pests canopy multi-source image dataset. Secondly, a vegetation index selection method based on saliency attention is proposed, which uses a multi-label ReliefF feature selection algorithm to obtain the importance scores of vegetation indices, enabling the automatic selection of vegetation indices. Finally, an apple disease and pest area multi-label classification model named AMMFNet is constructed, which effectively combines the advantages of RGB and multispectral multi-source images, performs data-level fusion of multi-source image data, and combines channel attention mechanisms to exploit the complementary aspects between multi-source data. The experimental results demonstrated that the proposed AMMFNet achieves a significant subset accuracy of 92.92%, a sample accuracy of 85.43%, and an F1 value of 86.21% on the apple disease and pest multi-source image dataset, representing improvements of 8.93% and 10.9% compared to prediction methods using only RGB or multispectral images. The experimental results also proved that the proposed method can provide technical support for the coarse-grained positioning of diseases and pests in apple orchards and has good application potential in the apple planting industry.

Multi-Label Emotion Recognition of Korean Speech Data Using Deep Fusion Models

As speech is the most natural way for humans to express emotions, studies on Speech Emotion Recognition (SER) have been conducted in various ways However, there are some areas for improvement in previous SER studies: (1) while some studies have performed multi-label classification, almost none have specifically utilized Korean speech data; (2) most studies have not utilized multiple features in combination for emotion recognition. Therefore, this study proposes deep fusion models for multi-label emotion classification using Korean speech data and follows four steps: (1) preprocessing speech data labeled with Sadness, Happiness, Neutral, Anger, and Disgust; (2) applying data augmentation to address the data imbalance and extracting speech features, including the Log-mel spectrogram, Mel-Frequency Cepstral Coefficients (MFCCs), and Voice Quality Features; (3) constructing models using deep fusion architectures; and (4) validating the performance of the constructed models. The experimental results demonstrated that the proposed model, which utilizes the Log-mel spectrogram and MFCCs with a fusion of Vision-Transformer and 1D Convolutional Neural Network–Long Short-Term Memory, achieved the highest average binary accuracy of 71.2% for multi-label classification, outperforming other baseline models. Consequently, this study anticipates that the proposed model will find application based on Korean speech, specifically mental healthcare and smart service systems.

The comparative experimental study of rehabilitation program decision for spinal cord injury based on electronic medical records

ObjectiveElectronic medical records (EMRs) contain patients' medical and health information. The Utilization of EMRs for assisted diagnosis is of significant importance for the rehabilitation of spinal cord injury (SCI) patients. Therefore, this study proposes a decision-making model for rehabilitation programs of SCI patients based on EMRs. MethodsFirst, an Electronic Medical Records (EMR) dataset comprising 1252 Spinal Cord Injury (SCI) patients was constructed, and data preprocessing was completed. Second, the Random Forest (RF) feature extraction algorithm was utilized to select case features with high contribution levels. Then, to address the imbalance issue in EMRs, a multi-label learning framework based on the improved MLSMOTE was adopted. Finally, seven multi-label classification models were employed to predict patients' physical therapy (PT) prescriptions. ResultsThe proposed improved MLSMOTE multi-label learning framework can solve the problem of class imbalance. Compared with the other six models, the CC model has improved significantly in many metrics. Its hamming loss and ranking loss were 0.1388 and 0.2467, and precision, recall, and F1-score were 83.33 %, 81.20 %, and 79.82 % respectively. ConclusionsThe improved MLSMOTE multi-label learning framework proposed in this study can make full use of the information in EMRs and effectively improve the decision-making accuracy of rehabilitation treatment programs.

A Lightweight Multi-Label Classification Method for Urban Green Space in High-Resolution Remote Sensing Imagery

Urban green spaces are an indispensable part of the ecology of cities, serving as the city’s “purifier” and playing a crucial role in promoting sustainable urban development. Therefore, the refined classification of urban green spaces is an important task in urban planning and management. Traditional methods for the refined classification of urban green spaces heavily rely on expert knowledge, often requiring substantial time and cost. Hence, our study presents a multi-label image classification model based on MobileViT. This model integrates the Triplet Attention module, along with the LSTM module, to enhance its label prediction capabilities while maintaining its lightweight characteristic for standalone operation on mobile devices. Trial outcomes in our UGS dataset in this study demonstrate that the approach we used outperforms the baseline by 1.64%, 3.25%, 3.67%, and 2.71% in mAP,F1,precision, and recall, respectively. This indicates that the model can uncover the latent dependencies among labels to enhance the multi-label image classification device’s performance. This study provides a practical solution for the intelligent and detailed classification of urban green spaces, which holds significant importance for the management and planning of urban green spaces.

A model for identifying potentially inappropriate medication used in older people with dementia: a machine learning study.

Older adults with dementia often face the risk of potentially inappropriate medication (PIM) use. The quality of PIM evaluation is hindered by researchers' unfamiliarity with evaluation criteria for inappropriate drug use. While traditional machine learning algorithms can enhance evaluation quality, they struggle with the multilabel nature of prescription data. This study aimed to combine six machine learning algorithms and three multilabel classification models to identify correlations in prescription information and develop an optimal model to identify PIMs in older adults with dementia. This study was conducted from January 1, 2020, to December 31, 2020. We used cluster sampling to obtain prescription data from patients 65 yearsand older with dementia. We assessed PIMs using the 2019 Beers criteria, the most authoritative and widely recognized standard for PIM detection. Our modeling process used three problem transformation methods (binary relevance, label powerset, and classifier chain) and six classification algorithms. We identified 18,338 older dementia patients and 36 PIMs types. The classifier chain + categorical boosting (CatBoost) model demonstrated superior performance, with the highest accuracy (97.93%), precision (95.39%), recall (94.07%), F1 score (95.69%), and subset accuracy values (97.41%), along with the lowest Hamming loss value (0.0011) and an acceptable duration of the operation (371s). This research introduces a pioneering CC + CatBoost warning model for PIMs in older dementia patients, utilizing machine-learning techniques. This model enables a quick and precise identification of PIMs, simplifying the manual evaluation process.

Explainable Knowledge Distillation for On-Device Chest X-Ray Classification.

Automated multi-label chest X-rays (CXR) image classification has achieved substantial progress in clinical diagnosis via utilizing sophisticated deep learning approaches. However, most deep models have high computational demands, which makes them less feasible for compact devices with low computational requirements. To overcome this problem, we propose a knowledge distillation (KD) strategy to create the compact deep learning model for the real-time multi-label CXR image classification. We study different alternatives of CNNs and Transforms as the teacher to distill the knowledge to a smaller student. Then, we employed explainable artificial intelligence (XAI) to provide the visual explanation for the model decision improved by the KD. Our results on three benchmark CXR datasets show that our KD strategy provides the improved performance on the compact student model, thus being the feasible choice for many limited hardware platforms. For instance, when using DenseNet161 as the teacher network, EEEA-Net-C2 achieved an AUC of 83.7%, 87.1%, and 88.7% on the ChestX-ray14, CheXpert, and PadChest datasets, respectively, with fewer parameters of 4.7 million and computational cost of 0.3 billion FLOPS.

Comprehensive comparisons of gradient-based multi-label adversarial attacks

Adversarial examples which mislead deep neural networks by adding well-crafted perturbations have become a major threat to classification models. Gradient-based white-box attack algorithms have been widely used to generate adversarial examples. However, most of them are designed for multi-class models, and only a few gradient-based adversarial attack algorithms specifically designed for multi-label classification models. Due to the correlation between multiple labels, the performance of these gradient-based algorithms in generating adversarial examples for multi-label classification is worthy of analyzing and evaluating comprehensively. In this paper, we first transplant five typical gradient-based adversarial attack algorithms in the multi-class environment to the multi-label environment. Secondly, we comprehensively compared the performance of these five attack algorithms and the other four existing multi-label adversarial attack algorithms by experiments on six different attack types, and evaluated the transferability of adversarial examples generated by all algorithms under two attack types. Experimental results show that, among different attack types, the majority of multi-step attack algorithms have higher attack success rates compared to one-step attack algorithms. Additionally, these gradient-based algorithms face greater difficulty in augmenting labels than in hiding them. For transfer experimental results, the adversarial examples generated by all attack algorithms exhibit weaker transferability when attacking other different models.

HTC-retina: A hybrid retinal diseases classification model using transformer-Convolutional Neural Network from optical coherence tomography images

Retinal diseases are among nowadays major public health issues, deservedly needing advanced computer-aided diagnosis. We propose a hybrid model for multi label classification, whereby seven retinal diseases are automatically classified from Optical Coherence Tomography (OCT) images. We show that, by combining the strengths of Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs), we can produce a more powerful type of model for medical image classification, especially when considering local lesion information such as retinal diseases. CNNs are indeed proved to be efficient at parameter utilization and provide the ability to extract local features and multi-scale feature maps through convolutional operations. On the other hand, ViT’s self-attention procedure allows processing long-range and global dependencies within an image. The paper clearly shows that the hybridization of these complementary capabilities (CNNs-ViTs) presents a high image processing potential that is more robust and efficient. The proposed model adopts a hierarchical CNN module called Convolutional Patch and Token Embedding (CPTE) instead of employing a direct tokenization approach using the raw input OCT image in the transformer. The CPTE module’s role is to incorporate an inductive bias, to reduce the reliance on large-scale datasets, and to address the low-level feature extraction challenges of the ViT. In addition, considering the importance of local lesion information in OCT images, the model relies on a parallel module called Residual Depthwise-Pointwise ConvNet (RDP-ConvNet) for extracting high-level features. RDP-ConvNet utilizes depthwise and pointwise convolution layers within a residual network architecture. The overall performance of the HTC-Retina model was evaluated on three datasets: the OCT-2017, OCT-C8, and OCT-2014 ; outperforming previous established models, achieving accuracy rates of 99.40%, 97.00%, and 99.77%, respectively ; and sensitivity rates of 99.41%, 97.00%, and 99.77%, respectively. Notably, the model showed high performance while maintaining computational efficiency.

CSAMDT: Conditional Self Attention Memory-Driven Transformers for Radiology Report Generation from Chest X-Ray.

A radiology report plays a crucial role in guiding patient treatment, but writing these reports is a time-consuming task that demands a radiologist's expertise. In response to this challenge, researchers in the subfields of artificial intelligence for healthcare have explored techniques for automatically interpreting radiographic images and generating free-text reports, while much of the research on medical report creation has focused on image captioning methods without adequately addressing particular report aspects. This study introduces a Conditional Self Attention Memory-Driven Transformer model for generating radiological reports. The model operates in two phases: initially, a multi-label classification model, utilizing ResNet152 v2 as an encoder, is employed for feature extraction and multiple disease diagnosis. In the second phase, the Conditional Self Attention Memory-Driven Transformer serves as a decoder, utilizing self-attention memory-driven transformers to generate text reports. Comprehensive experimentation was conducted to compare existing and proposed techniques based on Bilingual Evaluation Understudy (BLEU) scores ranging from 1 to 4. The model outperforms the other state-of-the-art techniques by increasing the BLEU 1 (0.475), BLEU 2 (0.358), BLEU 3 (0.229), and BLEU 4 (0.165) respectively. This study's findings can alleviate radiologists' workloads and enhance clinical workflows by introducing an autonomous radiological report generation system.

Convolution Neural Network Based Multi-Label Disease Detection Using Smartphone Captured Tongue Images

Purpose: Tongue image analysis for disease diagnosis is an ancient, traditional, non-invasive diagnostic technique widely used by traditional medicine practitioners. Deep learning-based multi-label disease detection models have tremendous potential for clinical decision support systems because they facilitate preliminary diagnosis. Methods: In this work, we propose a multi-label disease detection pipeline where observation and analysis of tongue images captured and received via smartphones assist in predicting the health status of an individual. Subjects, who consult collaborating physicians, voluntarily provide all images. Images thus acquired are first and foremost classified either into a diseased or a normal category by a 5-fold cross-validation algorithm using a convolutional neural network (MobileNetV2) model for binary classification. Once it predicts the diseased label, the disease prediction algorithm based on DenseNet-121 uses the image to diagnose single or multiple disease labels. Results: The MobileNetV2 architecture-based disease detection model achieved an average accuracy of 93% in distinguishing between diseased and normal, healthy tongues, whereas the multilabel disease classification model produced more than 90% accurate results for the disease class labels considered, strongly indicating a successful outcome with the smartphone-captured image dataset. Conclusion: AI-based image analysis shows promising results, and an extensive dataset could provide further improvements to this approach. Experimenting with smartphone images opens a great opportunity to provide preliminary health status to individuals at remote locations as well, prior to further treatment and diagnosis, using the concept of telemedicine.

Use of Natural Language Processing to Infer Sites of Metastatic Disease From Radiology Reports at Scale.

To evaluate natural language processing (NLP) methods to infer metastatic sites from radiology reports. A set of 4,522 computed tomography (CT) reports of 550 patients with 14 types of cancer was used to fine-tune four clinical large language models (LLMs) for multilabel classification of metastatic sites. We also developed an NLP information extraction (IE) system (on the basis of named entity recognition, assertion status detection, and relation extraction) for comparison. Model performances were measured by F1 scores on test and three external validation sets. The best model was used to facilitate analysis of metastatic frequencies in a cohort study of 6,555 patients with 53,838 CT reports. The RadBERT, BioBERT, GatorTron-base, and GatorTron-medium LLMs achieved F1 scores of 0.84, 0.87, 0.89, and 0.91, respectively, on the test set. The IE system performed best, achieving an F1 score of 0.93. F1 scores of the IE system by individual cancer type ranged from 0.89 to 0.96. The IE system attained F1 scores of 0.89, 0.83, and 0.81, respectively, on external validation sets including additional cancer types, positron emission tomography-CT ,and magnetic resonance imaging scans, respectively. In our cohort study, we found that for colorectal cancer, liver-only metastases were higher in de novo stage IV versus recurrent patients (29.7% v 12.2%; P < .001). Conversely, lung-only metastases were more frequent in recurrent versus de novo stage IV patients (17.2% v 7.3%; P < .001). We developed an IE system that accurately infers metastatic sites in multiple primary cancers from radiology reports. It has explainable methods and performs better than some clinical LLMs. The inferred metastatic phenotypes could enhance cancer research databases and clinical trial matching, and identify potential patients for oligometastatic interventions.