Pre-trained Model Research Articles

Adaptive genetic algorithm based deep feature selector for cancer detection in lung histopathological images

Cancer is a global health concern because of a significant mortality rate and a wide range of affected organs. Early detection and accurate classification of cancer types are crucial for effective treatment. Imaging tests on different image modalities such as Histopathology images, provide valuable insights into the cellular and architectural features of tissues, allowing pathologists to make diagnosis, determine disease stages, and guide treatment decisions. They are an essential tool in the study and understanding of diseases, aiding in research, education, and patient care. Convolutional neural network based pretrained deep learning models can be used successfully to detect lung cancer. In this study, we have used a channel attention-enabled deep learning model as a feature extractor followed by an adaptive Genetic Algorithm (GA) based feature selector. Here, we calculate the fitness score of each chromosome (i.e., a candidate solution) using a filter method, instead of a classifier. Further, the GA optimized feature vector is fed to the K-nearest neighbors classifier for final classification. The proposed method shows a promising result with an overall accuracy of 99.75% on the LC25000 dataset, which is a publicly available dataset of lung histopathological images. The source code for this work can be found https://github.com/priyam-03/GA-Feature-Selector-Lung-Cancer.

A monocular thoracoscopic 3D scene reconstruction framework based on NeRF.

With the increasing use of image-based 3D reconstruction in medical procedures, accurate scene reconstruction plays a crucial role in surgical navigation and assisted treatment. However, the monotonous colors, limited image features, and obvious brightness fluctuations of thoracoscopic scenes make the feature point matching process, on which traditional 3D reconstruction methods rely, unstable and unreliable. It brings a great challenge to accurate 3D reconstruction. In this study, a new method for implicit 3D reconstruction of monocular thoracoscopic scenes is proposed. The method combines a pre-trained metric depth estimation model with neural radiation field (NeRF) technique and uses dense SLAM to accurately compute the camera pose. To ensure the accuracy of the depth values and the structural consistency of the reconstructed scene, depth and normal constraints are added to the original color constraints of the NeRF network to achieve high-quality scene reconstruction results. We conducted experiments on the SCARED dataset and the clinical dataset. After comparing with other methods, the depth estimation accuracy and point cloud reconstruction quality of this paper outperform the existing methods. The method in this paper can provide more accurate 3D reconstruction of complex thoracic surgical scenes, which can significantly improve the accuracy and therapeutic efficacy of surgical navigation.

K-Bloom: unleashing the power of pre-trained language models in extracting knowledge graph with predefined relations

K-Bloom: unleashing the power of pre-trained language models in extracting knowledge graph with predefined relations

An automatic pipeline for temporal monitoring of radiotherapy-induced toxicities in head and neck cancer patients

Radiotherapy for head and neck cancer often causes a spectrum of toxicities. Such toxicities are usually unavailable as structured data and are reported within textual clinical reports. To reduce the burden of manual assessment of toxicities, we propose a language processing model for the automatic extraction of toxicities. The cohort consists of 384 patients with head and neck cancer who underwent radiotherapy, either as monotherapy or in combination with chemotherapy or surgery. A total of 3510 notes were extracted. The toxicities were then manually annotated. Two tasks of toxicity mention detection and toxicity extraction were defined. Pre-trained language models such as BERT, Clinical BioBERT, and Clinical Longformer were fine-tuned. Our best model achieves an F1 score of 90% for automatic extraction of toxicity mentions. An automatic system enables real-time extraction of toxicities and insights into their temporal patterns, offering actionable data to support dose optimization and minimize toxicities in personalized treatments.

Detection of oscillation-like patterns in eclipsing binary light curves using neural network-based object detection algorithms

The primary aim of this research is to evaluate several convolutional neural network-based object detection algorithms for identifying oscillation-like patterns in light curves of eclipsing binaries. This involved creating a robust detection framework that can effectively process both synthetic light curves and real observational data. The study employs several state-of-the-art object detection algorithms, including Single Shot MultiBox Detector, Faster Region-based Convolutional Neural Network, You Only Look Once, and EfficientDet, as well as a custom non-pretrained model implemented from scratch. Synthetic light curve images and images derived from observational TESS light curves of known eclipsing binaries with a pulsating component were constructed with corresponding annotation files using custom scripts. The models were trained and validated on established datasets, which was followed by testing on unseen Kepler data to assess their generalisation performance. The statistical metrics were also calculated to review the quality of each model. The results indicate that the pre-trained models exhibit high accuracy and reliability in detecting the targeted patterns. The Faster Region-based Convolutional Neural Network and You Only Look Once in particular showed superior performance in terms of object detection evaluation metrics on the validation dataset, including a mean average precision value exceeding 99%. The Single Shot MultiBox Detector, on the other hand, is the fastest, although it shows a slightly lower performance, with a mean average precision of 97%. These findings highlight the potential of these models to significantly contribute to the automated determination of pulsating components in eclipsing binary systems and thus facilitate more efficient and comprehensive astrophysical investigations.

Alignclip: navigating the misalignments for robust vision-language generalization

In the realm of Vision-Language Pretraining models, achieving robust and adaptive representations is a cornerstone for successfully handling the unpredictability of real-world scenarios. This paper delves into two pivotal misalignment challenges inherent to Contrastive Language-Image Pre-training (CLIP) models: attention misalignment, which leads to an overemphasis on background elements rather than salient objects, and predictive category misalignment, characterized by the model’s struggle to discern between classes based on similarity. These misalignments undermine the representational stability essential for dynamic, real-world applications. To address these challenges, we propose AlignCLIP, an advanced fine-tuning methodology distinguished by its attention alignment loss, designed to calibrate the distribution of attention across multi-head attention layers. Furthermore, AlignCLIP introduces semantic label smoothing, a technique that leverages textual class similarities to refine prediction hierarchies. Through comprehensive experimentation on a variety of datasets and in scenarios involving distribution shifts and unseen classes, we demonstrate that AlignCLIP significantly enhances the stability of representations and shows superior generalization capabilities.

Instance-level semantic segmentation of nuclei based on multimodal structure encoding

BackgroundAccurate segmentation and classification of cell nuclei are crucial for histopathological image analysis. However, existing deep neural network-based methods often struggle to capture complex morphological features and global spatial distributions of cell nuclei due to their reliance on local receptive fields.MethodsThis study proposes a graph neural structure encoding framework based on a vision-language model. The framework incorporates: (1) A multi-scale feature fusion and knowledge distillation module utilizing the Contrastive Language-Image Pre-training (CLIP) model’s image encoder; (2) A method to transform morphological features of cells into textual descriptions for semantic representation; and (3) A graph neural network approach to learn spatial relationships and contextual information between cell nuclei.ResultsExperimental results demonstrate that the proposed method significantly improves the accuracy of cell nucleus segmentation and classification compared to existing approaches. The framework effectively captures complex nuclear structures and global distribution features, leading to enhanced performance in histopathological image analysis.ConclusionsBy deeply mining the morphological features of cell nuclei and their spatial topological relationships, our graph neural structure encoding framework achieves high-precision nuclear segmentation and classification. This approach shows significant potential for enhancing histopathological image analysis, potentially leading to more accurate diagnoses and improved understanding of cellular structures in pathological tissues.

CaBind_MCNN: Identifying Potential Calcium Channel Blocker Targets by Predicting Calcium-Binding Sites in Ion Channels and Ion Transporters Using Protein Language Models and Multiscale Feature Extraction.

Calcium ions (Ca2+) are crucial for various physiological processes, including neurotransmission and cardiac function. Dysregulation of Ca2+ homeostasis can lead to serious health conditions such as cardiac arrhythmias and hypertension. Ion channels and transporters play a vital role in maintaining cellular Ca2+ balance by facilitating Ca2+ transport across cell membranes. Accurate prediction of Ca2+ binding sites within these proteins is essential for understanding their function and identifying potential therapeutic targets, particularly for developing novel calcium channel blockers (CCBs). This study introduces CaBind_MCNN, an innovative computational model that leverages pretrained protein language models (PLMs) and a multiscale feature extraction approach to predict Ca2+ binding sites in ion channels and transporter proteins. Our method integrates embeddings from the ProtTrans PLM with a convolutional neural network (CNN)-based multiwindow scanning approach, capable of capturing diverse sequence features relevant to Ca2+ binding. The model, trained on a curated data set of 27 calcium-binding protein sequences, achieves high accuracy with an area under the curve (AUC) of 0.9886, significantly outperforming some existing methods. These results demonstrate the potential of CaBind_MCNN to enhance drug discovery efforts by identifying potential CCB targets and advancing the development of novel therapies for calcium-related disorders.

Targeted generative data augmentation for automatic metastases detection from free-text radiology reports

Automatic identification of metastatic sites in cancer patients from electronic health records is a challenging yet crucial task with significant implications for diagnosis and treatment. In this study, we demonstrate how advancements in natural language processing, namely the instruction-following capability of recent large language models and extensive model pretraining, made it possible to automate metastases detection from radiology reports texts with a limited amount of gold-labeled data. Specifically, we prompt Llama3, an open-source instruction-tuned large language model, to generate synthetic training data to expand our limited labeled data and adapt BERT, a small pretrained language model, to the task. We further investigate three targeted data augmentation techniques which selectively expand the original training samples, leading to comparable or superior performance compared to vanilla data augmentation, in most cases, while being substantially more computationally efficient. In our experiments, data augmentation improved the average F1-score by 2.3, 3.5, and 3.9 points for lung, liver, and adrenal glands, the organs for which we had access to expert-annotated data. This observation suggests that Llama3, which has not been specifically tailored to this task or clinical data in general, can generate high-quality synthetic data through paraphrasing in the clinical context. We also compare metastasis identification accuracy between models utilizing institutionally standardized reports vs. non-structured reports, which complicate the extraction of relevant information, and show how including patient history with a customized model architecture narrows the gap between those two setups from 7.3 to 4.5 points on F1-score under LoRA tuning. Our work delivers a broadly applicable solution with remarkable performance that does not require model customization for each institution, making large-scale, low-cost spatio-temporal cancer progression pattern extraction possible.

WDRIV-Net: a weighted ensemble transfer learning to improve automatic type stratification of lumbar intervertebral disc bulge, prolapse, and herniation

The degeneration of the intervertebral discs in the lumbar spine is the common cause of neurological and physical dysfunctions and chronic disability of patients, which can be stratified into single—(e.g., disc herniation, prolapse, or bulge) and comorbidity-type degeneration (e.g., simultaneous presence of two or more conditions), respectively. A sample of lumbar magnetic resonance imaging (MRI) images from multiple clinical hospitals in China was collected and used in the proposal assessment. We devised a weighted transfer learning framework WDRIV-Net by ensembling four pre-trained models including Densenet169, ResNet101, InceptionV3, and VGG19. The proposed approach was applied to the clinical data and achieved 96.25% accuracy, surpassing the benchmark ResNet101 (87.5%), DenseNet169 (82.5%), VGG19 (88.75%), InceptionV3 (93.75%), and other state-of-the-art (SOTA) ensemble deep learning models. Furthermore, improved performance was observed as well for the metric of the area under the curve (AUC), producing a ≥ 7% increase versus other SOTA ensemble learning, a ≥ 6% increase versus most-studied models, and a ≥ 2% increase versus the baselines. WDRIV-Net can serve as a guide in the initial and efficient type screening of complex degeneration of lumbar intervertebral discs (LID) and assist in the early-stage selection of clinically differentiated treatment options.

CART-ANOVA-Based Transfer Learning Approach for Seven Distinct Tumor Classification Schemes with Generalization Capability

Background/Objectives: Deep transfer learning, leveraging convolutional neural networks (CNNs), has become a pivotal tool for brain tumor detection. However, key challenges include optimizing hyperparameter selection and enhancing the generalization capabilities of models. This study introduces a novel CART-ANOVA (Cartesian-ANOVA) hyperparameter tuning framework, which differs from traditional optimization methods by systematically integrating statistical significance testing (ANOVA) with the Cartesian product of hyperparameter values. This approach ensures robust and precise parameter tuning by evaluating the interaction effects between hyperparameters, such as batch size and learning rate, rather than relying solely on grid or random search. Additionally, it implements seven distinct classification schemes for brain tumors, aimed at improving diagnostic accuracy and robustness. Methods: The proposed framework employs a ResNet18-based knowledge transfer learning (KTL) model trained on a primary dataset, with 20% allocated for testing. Hyperparameters were optimized using CART-ANOVA analysis, and statistical validation ensured robust parameter selection. The model’s generalization and robustness were evaluated on an independent second dataset. Performance metrics, including precision, accuracy, sensitivity, and F1 score, were compared against other pre-trained CNN models. Results: The framework achieved exceptional testing accuracy of 99.65% for four-class classification and 98.05% for seven-class classification on the source 1 dataset. It also maintained high generalization capabilities, achieving accuracies of 98.77% and 96.77% on the source 2 datasets for the same tasks. The incorporation of seven distinct classification schemes further enhanced variability and diagnostic capability, surpassing the performance of other pre-trained models. Conclusions: The CART-ANOVA hyperparameter tuning framework, combined with a ResNet18-based KTL approach, significantly improves brain tumor classification accuracy, robustness, and generalization. These advancements demonstrate strong potential for enhancing diagnostic precision and informing effective treatment strategies, contributing to advancements in medical imaging and AI-driven healthcare solutions.

Personalizing Multimedia Content Recommendations for Intelligent Vehicles Through Text–Image Embedding Approaches

The ability to automate and personalize the recommendation of multimedia contents to consumers has been gaining significant attention recently. The burgeoning demand for digitization and automation of formerly analog communication processes has caught the attention of researchers and professionals alike. In light of the recent interest and anticipated transition to fully autonomous vehicles, this study proposes a text–image embedding method recommender system for the optimization of personalized multimedia content for in-vehicle infotainment. This study leverages existing pre-trained text embedding models and pre-trained image feature extraction methods. Previous research to date has focused mainly on textual-only or image-only analyses. By employing similarity measurements, this study demonstrates how recommendation of the most relevant multimedia content to consumers is enhanced through text–image embedding.

A Hybrid Transfer Learning Framework for Brain Tumor Diagnosis

Brain tumors are among the most severe health challenges, necessitating early and precise diagnosis for effective treatment planning. This study introduces an optimized hybrid transfer learning (TL) framework for brain tumor classification using magnetic resonance imaging images. The proposed system integrates advanced preprocessing techniques, an ensemble of pretrained deep learning models, and explainable artificial intelligence (XAI) methods to achieve high accuracy and reliability. The methodology enhances image quality through noise reduction and contrast enhancement, facilitating robust feature extraction. The ensemble model combines VGG16 and ResNet152V2 architectures, achieving a classification accuracy of 99.47% on a challenging four‐class dataset. Additionally, gradient‐weighted class activation mapping and SHapley Additive exPlanations (SHAP)‐based XAI techniques provide visual and quantitative insights into model predictions, improving interpretability and clinical trust. This comprehensive framework demonstrates the potential of hybrid TL and XAI in advancing diagnostic accuracy and supporting clinical decision‐making for brain tumor detection. The results underscore its applicability in clinical settings, particularly in resource‐constrained environments.

Developing an artificial intelligence model for English grammar correction: A computational linguistics approach

This article aimed to explore the development of an artificial intelligence model for English grammar correction based on computational linguistics methods. Traditional grammar correction systems suffer from problems such as complex rules, sparse data, and insufficient utilization of contextual information. To address these issues, this article adopted Transformer’s pre-trained language model, utilizing its powerful context understanding and automatic feature extraction capabilities to improve the processing performance of complex syntax structures and long-distance dependencies. Meanwhile, by constructing large-scale and diverse datasets and combining them with data augmentation techniques, the model’s generalization ability and robustness were enhanced. This article also investigated hyperparameter tuning, model integration, and continuous optimization strategies in the process of model training optimization, and provided a detailed description of model evaluation and experimental validation. In the evaluation, the average precision, average recall, and average F1 score for most common grammar errors were 0.8, 0.805, and 0.801, respectively. The model in this article has excellent grammar correction ability. Through comprehensive experiments and evaluations, the potential and advantages of a new grammar correction artificial intelligence model developed based on computational linguistics methods in improving grammar correction effectiveness and practicality have been demonstrated.

Supervised contrastive pre-training models for mammography screening

Breast cancer is now the most deadly cancer worldwide. Mammography screening is the most effective method for early detection and diagnosis of breast cancer. Due to the lack of labeled mammograms, building an AI system for mammography screening often relies heavily on human-designed data augmentation, which doesn’t always perform robustly when applied to clinical scenarios. This paper presents a novel framework of Supervised Contrastive Pre-training followed by Supervised Fine-tuning (SCP+SF) for mammography screening. Unlike the previous approaches, the proposed supervised contrastive pre-training does not need a data augmentation module. We apply the SCP+SF framework to two challenging and important mammography screening tasks for breast cancer: mammographic abnormality screening and mammographic malignancy screening. Our extensive experiments on a large-scale dataset show that the supervised contrastive pre-training (SCP) can substantially improve the final model performance compared with the traditional direct supervised training approach. Superior results of AUC and specificity/sensitivity have been achieved on two clinically significant mammographic screening tasks in comparison with previously reported State-Of-The-Art approaches. We believe this work is the first to show that supervised contrastive pre-training (SCP) followed by supervised fine-tuning (SF) can outperform the supervised counterpart on these two critical medical imaging tasks.

CyclicFL: Efficient Federated Learning with Cyclic Model Pre-Training

Federated learning (FL) has been proposed to enable distributed learning on artificial intelligence Internet of Things (AIoT) devices with guarantees of high-level data privacy. Since random initial models in FL can easily result in unregulated stochastic gradient descent (SGD) processes, existing FL methods greatly suffer from both slow convergence and poor accuracy, especially in non-IID scenarios. To address this problem, we propose a novel method named CyclicFL, which can quickly derive effective initial models to guide the SGD processes, thus improving the overall FL training performance. We formally analyze the significance of data consistency between the pre-training and training stages of CyclicFL, showing the limited Lipschitzness of loss for the pre-trained models by CyclicFL. Moreover, we systematically prove that our method can achieve a faster convergence speed under various convexity assumptions. Unlike traditional centralized pre-training methods that require public proxy data, CyclicFL pre-trains initial models on selected AIoT devices cyclically without exposing their local data. Therefore, they can be easily integrated into any security-critical FL methods. Comprehensive experimental results show that CyclicFL can not only improve maximum classification accuracy by up to [Formula: see text], but also significantly accelerate the overall FL training process.

A Novel End-To-End Learning Framework Based on Optimised Residual Gated Units and Stacked Pre-Trained Layers for Detection of Autism Spectrum Disorders (ASD)

<p dir="ltr"><span>Autism Spectrum Disorder (ASD) is distinguished in detailed with diverse assemblage neural behaviours, diagnosed over the recent time of years. These ASD are developed using the increased internal connectivity among neural units of functional brain. Since Autism characteristics tend to occur over the months or even years, cognitive system for an early diagnosis of ASD is badly required to overcome the neural complexities of ASD with an accurate prediction. Electroencephalogram (EEG) is common technique for analysing brain’s electrical activity and widely used as the major tool for the gathering the neural information. Recent ensemble of EEG and Artificial Intelligence (AI) has thrown the brighter light in the ASD early prediction, but still computational overhead hurdles in achieving the highest performance. In this research article, novel deep learning framework that consist of hybrid combination of multiple tiers of pre-trained convolutional models and residual gated recurrent units with the hyper-parameters are tuned by the chaotic Cat swarm optimisation. As the first step, EEG signals are collected and translated into two-dimensional (2D) spectrograms followed by the pre-processing and feature engineering using the multi-scale stacked pre-trained models. Finally, the residual gated units are entangled for extracting the temporal features which are used to train the optimised classification networks. The proposed model is validated using EEG signals extracted from the 43 subjects and performance including accuracy, precision, recall and F1-score are measured then juxtaposed through the other cutting-edge learning models. The accuracy of detection is found to be 98.5%, precision was 98.2% recall was 98% and F1-score was 98% for the real time EEG subjects and also statistically proved its stability in handling the non-linear EEG datasets.</span></p>

I-Brainer: Artificial intelligence/Internet of Things (AI/IoT)-Powered Detection of Brain Cancer.

Brain tumour is characterized by its aggressive nature and low survival rate and thus regarded as one of the deadliest diseases. Thus, miss-diagnosis or miss-classification of brain tumour can lead to miss treatment or incorrect treatment and reduce survival chances. Therefore, there is need to develop a technique that can identify and detect brain tumour at early stages. Here, we proposed a framework titled I-Brainer which is an Artificial Intelligence/Internet of Things (AI/IoT)-powered classification of MRI. We employed a Br35H+SARTAJ brain MRI dataset which contain 7023 total images which include No tumour, pituitary, meningioma and glioma. In order to accurately classified MRI into 4-class, we developed LeNet model from scratch, implemented 2 pretrained models which include EfficientNet and ResNet-50 as well feature extraction of these models coupled with 2 Machine Learning classifiers k-Nearest Neighbours (KNN) and Support Vector Machines (SVM). Evaluation and comparison of the performance of 3 models has shown that EfficientNet+SVM achieved the best result in terms of AUC (99%) and ResNet-50-KNN ranked higher in terms of accuracy (94%) on testing dataset. This framework can be harness by patients residing in remote areas and as confirmatory approach for medical experts.

Advancing Pancreatic Cancer Prediction with a Next Visit Token Prediction Head on Top of Med-BERT

Background: Electronic Health Records (EHRs) encompass valuable data essential for disease prediction. The application of artificial intelligence (AI), particularly deep learning, significantly enhances disease prediction by analyzing extensive EHR datasets to identify hidden patterns, facilitating early detection. Recently, numerous foundation models pretrained on extensive data have demonstrated efficacy in disease prediction using EHRs. However, there remains some unanswered questions on how to best utilize such models, especially with very small fine-tuning cohorts. Methods: We utilized Med-BERT, an EHR-specific foundation model, and reformulated the disease binary prediction task into a token prediction task and a next visit mask token prediction task to align with Med-BERT’s pretraining task format in order to improve the accuracy of pancreatic cancer (PaCa) prediction in both few-shot and fully supervised settings. Results: The reformulation of the task into a token prediction task, referred to as Med-BERT-Sum, demonstrated slightly superior performance in both few-shot scenarios and larger data samples. Furthermore, reformulating the prediction task as a Next Visit Mask Token Prediction task (Med-BERT-Mask) significantly outperformed the conventional Binary Classification (BC) prediction task (Med-BERT-BC) by 3% to 7% in few-shot scenarios with data sizes ranging from 10 to 500 samples. These findings highlight that aligning the downstream task with Med-BERT’s pretraining objectives substantially enhances the model’s predictive capabilities, thereby improving its effectiveness in predicting both rare and common diseases. Conclusions: Reformatting disease prediction tasks to align with the pretraining of foundation models enhances prediction accuracy, leading to earlier detection and timely intervention. This approach improves treatment effectiveness, survival rates, and overall patient outcomes for PaCa and potentially other cancers.

A comparative analysis of the binary and multiclass classified chest X-ray images of pneumonia and COVID-19 with ML and DL models

Abstract Background The highly infectious coronavirus disease 2019 (COVID-19) is caused by severe acute respiratory syndrome coronavirus 2, the seventh coronavirus. It is the longest pandemic in recorded history worldwide. Many countries are still reporting COVID-19 cases even in the fifth year of its emergence. Objective The performance of various machine learning (ML) and deep learning (DL) models was studied for image-based classification of the lungs infected with COVID-19, pneumonia (viral and bacterial), and normal cases from the chest X-rays (CXRs). Methods The K-nearest neighbour and logistics regression as the two ML models, and Visual Geometry Group-19, Vision transformer, and ConvMixer as the three DL models were included in the investigation to compare the brevity of the detection and classification of the cases. Results Among the investigated models, ConvMixer returned the best result in terms of accuracy, recall, precision, F1-score and area under the curve for both binary as well as multiclass classification. The pre-trained ConvMixer model outperformed the other four models in classifying. As per the performance observations, there was 97.1% accuracy for normal and COVID-19 + pneumonia-infected lungs, 98% accuracy for normal and COVID-19 infected lungs, 82% accuracy for normal + bacterial + viral infected lungs, and 98% accuracy for normal + pneumonia infected lungs. The DL models performed better than the ML models for binary and multiclass classification. The performance of these studied models was tried on other CXR image databases. Conclusion The suggested network effectively detected COVID-19 and different types of pneumonia by using CXR imagery. This could help medical sciences for timely and accurate diagnoses of the cases through bioimaging technology and the use of high-end bioinformatics tools.