Pre-trained Convolutional Neural Network Model Research Articles

Assessment of artificial intelligence to detect gasoline in fire debris using HS-SPME-GC/MS and transfer learning.

Due to the complex nature of the chemical compositions of ignitable liquids (IL) and the interferences from fire debris matrices, interpreting chromatographic data poses challenges to analysts. In this work, artificial intelligence (AI) was developed by transfer learning in a convolutional neural network (CNN), GoogLeNet. The image classification AI was fine-tuned to create intelligent classification systems to discriminate samples containing gasoline residues from burned substrates. All ground truth samples were analyzed by headspace solid-phase microextraction (HS-SPME) coupled with a gas chromatograph and mass spectrometer (GC/MS). The HS-SPME-GC/MS data were transformed into three types of image presentations, that is, heatmaps, extracted ion heatmaps, and total ion chromatograms. The abundance and mass-to-charge ratios of each scan were converted into image patterns that are characteristic of the chemical profiles of gasoline. The transfer learning data were labeled as "gasoline present" and "gasoline absent" classes. The assessment results demonstrated that all AI models achieved 100 ± 0% accuracy in identifying neat gasoline. When the models were assessed using the spiked samples, the AI model developed using the extracted ion heatmap obtained the highest accuracy rate (95.9 ± 0.4%), which was greater than those obtained by other machine learning models, ranging from 17.3 ± 0.7% to 78.7 ± 0.7%. The proposed work demonstrated that the heatmaps created from GC/MS data can represent chemical features from the samples. Additionally, the pretrained CNN models are readily available in the transfer learning workflow to develop AI for GC/MS data interpretation in fire debris analysis.

A Novel Hybrid Approach Based on CNN for Corn Diseases Detection

Corn is one of the most economically important crops globally, significantly improving food security and agricultural productivity. However, corn plants face various foliar diseases, which can significantly reduce crop productivity. Accurate and early detection of corn diseases is an imperative task for maintaining crop health and ensuring food security. In this study, we propose a novel approach for corn disease detection by integrating DenseNet121, a powerful convolutional neural network (CNN) architecture, with a deep neural network (DNN) classifier. This hybrid model, called DenseNetDNN, combines the feature extraction capabilities of DenseNet121 with the classification capabilities of a DNN, aiming to enhance disease detection accuracy. The proposed model’s performance is compared against four widely used pre-trained CNN models: ResNet50, MobileNet, EfficientNetB0, and Xception. All models are evaluated using accuracy, precision, and recall. Additionally, the study employs GradeCam, an advanced grading system, to automate and standardize the performance evaluation of the proposed model. Results demonstrate that the DenseNetDNN model outperformed all other models in terms of identifying corn diseases; it achieves superior performance with an accuracy of 96.1%, precession of 0.952, and recall of 0.958. which demonstrates the efficiency of DenseNetDNN in advancing agricultural disease detection. This research contributes to the development of automated solutions for agricultural monitoring, with implications for improving crop management practices and ensuring global food security.

Performance Enhancement of Alzheimer's Disease Diagnosis Using Generative Adversarial Network

Insufficient medical images have always been a challenge for deep learning-based Alzheimer's disease classification and detection tasks. The availability of Magnetic Resonance Imaging (MRI) data is limited due to patients' privacy issues. Access to individual medical records is strongly protected by the law, and appropriate consent is needed to utilize them for research purposes. Besides that, publicly available databases often experience imbalanced classes, further contributing to the issue of insufficient data. Thus, the performance of deep learning models used to diagnose Alzheimer's disease is often hindered by this issue. Basic data augmentation methods such as geometrical augmentation techniques also had limited applications to medical data. Hence, this study proposes a deep learning technique of Generative Adversarial Network (GAN) to expand the MRI dataset and improve the classification model performance. MRI images from the Open Access Series of Imaging Studies (OASIS) database are used in this study to perform experiments and validate hypotheses. After applying GAN to expand the dataset, a pre-trained Convolutional Neural Network (CNN) model is used to classify the data into multiple classes of Alzheimer's and the model's performance is measured. As a result, an improvement in accuracy for the classification task can be observed, indicating that the GAN is a solution for overcoming the challenge of insufficient data for Alzheimer's diagnosis.

Enhancing multimedia management: cloud-based movie type recognition with hybrid deep learning architecture

Film and movie genres play a pivotal role in captivating relevant audiences across interactive multimedia platforms. With a focus on entertainment, streaming providers are increasingly prioritizing the automatic generation of movie genres within cloud-based media services. In service management, the integration of a hybrid convolutional network proves to be instrumental in effectively distinguishing between a diverse array of video genres. This classification process not only facilitates more refined recommendations and content filtering but also enables targeted advertising. Furthermore, given the frequent amalgamation of components from various genres in cinema, there arises a need for social media networks to incorporate real-time video classification mechanisms for accurate genre identification. In this study, we propose a novel architecture leveraging deep learning techniques for the detection and classification of genres in video films. Our approach entails the utilization of a bidirectional long- and short-term memory (BiLSTM) network, augmented with video descriptors extracted from EfficientNet-B7, an ImageNet pre-trained convolutional neural network (CNN) model. By employing BiLSTM, the network acquires robust video representations and proficiently categorizes movies into multiple genres. Evaluation on the LMTD dataset demonstrates the substantial improvement in the performance of the movie genre classifier system achieved by our proposed architecture. Notably, our approach achieves both computational efficiency and precision, outperforming even the most sophisticated models. Experimental results reveal that EfficientNet-BiLSTM achieves a precision rate of 93.5%. Furthermore, our proposed architecture attains state-of-the-art performance, as evidenced by its F1 score of 0.9012.

Human Emotion Detection Using CNN and Transfer Learning

Facial emotion detection is a critical component in human-computer interaction, mental health assessment, and security systems. In this project, we propose a robust facial emotion detection system leveraging state-of-the-art deep learning techniques. Our system utilizes Convolutional Neural Networks (CNNs) to extract meaningful features from facial images and classify them into five distinct emotional categories: neutral, surprise, sad, happy, and angry. We conducted extensive experiments on a diverse dataset consisting of over 10,000 annotated facial images collected from various sources. Through data augmentation techniques such as rotation, translation, and flipping, we expanded the dataset to enhance model training. Additionally, we employed transfer learning by fine-tuning a pre-trained CNN model, ResNet50, on our dataset to leverage its learned features. This project presents a system for real-time emotion monitoring using computer vision techniques. The system utilizes the Haar Cascade Classifier for face detection in live webcam video streams. Furthermore, we evaluated the system's performance across different lighting conditions, poses, and facial occlusions to assess its robustness in real-world scenarios. Our results indicate that the system maintains consistent performance across diverse conditions, making it suitable for deployment in applications requiring real-time facial emotion recognition.

Stress Sense: Enhanced Stress Detection and Management Via Image Processing

This project presents an innovative approach to stress detection by utilizing Convolutional Neural Networks (CNNs) to analyze emotional cues extracted from facial images. The proposed system employs CNNs, a class of deep learning models known for their efficacy in image recognition tasks, to automatically extract features from facial images. Through a combination of convolutional, pooling, and fully connected layers, the CNN learns hierarchical representations of facial expressions associated with various emotions, including those indicative of stress. The model is trained on a diverse dataset encompassing a wide range of facial expressions, allowing it to generalize well to unseen data. Transfer learning techniques may also be employed to leverage pre-trained CNN models, further enhancing performance with limited data.

Automatic classification and segmentation of blast cells using deep transfer learning and active contours.

Acute lymphoblastic leukemia (ALL) presents a formidable challenge in hematological malignancies, necessitating swift and precise diagnostic techniques for effective intervention. The conventional manual microscopy of blood smears, although widely practiced, suffers from significant limitations including labor-intensity and susceptibility to human error, particularly in distinguishing the subtle differences between normal and leukemic cells. To overcome these limitations, our research introduces the ALLDet classifier, an innovative tool employing deep transfer learning for the automated analysis and categorization of ALL from White Blood Cell (WBC) nuclei images. Our investigation encompassed the evaluation of nine state-of-the-art pre-trained convolutional neural network (CNN) models, namely VGG16, VGG19, ResNet50, ResNet101, DenseNet121, DenseNet201, Xception, MobileNet, and EfficientNetB3. We augmented this approach by incorporating a sophisticated contour-based segmentation technique, derived from the Chan-Vese model, aimed at the meticulous segmentation of blast cell nuclei in blood smear images, thereby enhancing the accuracy of our analysis. The empirical assessment of these methodologies underscored the superior performance of the EfficientNetB3 model, which demonstrated exceptional metrics: a recall specificity of 98.5%, precision of 95.86%, F1-score of 97.16%, and an overall accuracy rate of 97.13%. The Chan-Vese model's adaptability to the irregular shapes of blast cells and its noise-resistant segmentation capability were key to capturing the complex morphological changes essential for accurate segmentation. The combined application of the ALLDet classifier, powered by EfficientNetB3, with our advanced segmentation approach, emerges as a formidable advancement in the early detection and accurate diagnosis of ALL. This breakthrough not only signifies a pivotal leap in leukemia diagnostic methodologies but also holds the promise of significantly elevating the standards of patient care through the provision of timely and precise diagnoses. The implications of this study extend beyond immediate clinical utility, paving the way for future research to further refine and enhance the capabilities of artificial intelligence in medical diagnostics.

Transfer learning-based quality monitoring of laser powder bed fusion across materials

Laser powder bed fusion (LPBF) is a promising additive manufacturing (AM) technique that has gained rapid growth. Despite the unique advantages such as producing complex-shaped metal parts, LPBF has been still slowed by the poor process repeatability and part consistency. In-situ monitoring methods combining sensing sensors and machine leaning (ML) tools have attracted much attention for defect detection and quality identification. However, the current ML algorithms usually rely on the availability of massive labeled datasets, and the trained model for monitoring a certain powder material cannot be directly applied to monitoring another material with different properties. Moreover, for a new material, it is time-consuming and expensive to collect the sufficient labeled data for building the quality monitoring model from scratch. To this end, motivated by transfer learning (TL), we propose a TL-based quality monitoring approach across two different materials in the LPBF process. Three different sensing datasets of layer-wise images of the solidified layer, photodiode signals, and acoustic emission signals are separately transferred from the 316L stainless steel to the TC4 titanium alloy. A convolutional neural network (CNN)-based quality monitoring model is first built by the sufficient source datasets from 316L stainless steel material. Then, the pre-trained CNN model is fine-tuned with a small amount of the target datasets for quality monitoring on TC4 titanium alloy material. Experimental results demonstrate the effectiveness of the proposed method for quality monitoring from one material to another with limited target datasets, with a classification accuracy above 92 % just with 5 % of target datasets on all three transferred datasets.

Large-scale Experiment in STEM Education for High School Students using Artificial Intelligence Kit Based on Computer Vision and Python

This study proposes an artificial intelligence (AI) kit for high school students in science, technology, engineering, and mathematics (STEM). The AI kit includes an edge AI machine and electronic components. A compact, purpose-built kit resembling a laptop was designed for ease of replication and portability. Utilizing pre-trained convolutional neural network models and computer vision algorithms, five Thai schools participated in on-site instructions. A quasi-experimental study assessed the students’ learning outcomes using a paired sample t-test. Results revealed improved knowledge and reduced score variation. Additionally, gender analysis confirmed that both male and female students met the learning criteria. The students expressed satisfaction with the distinctive hardware and learning method employed during the class activities. Notably, the test results demonstrated that the AI kit enhanced students’ enthusiasm and facilitated comprehension.

Automated detection of COVID-19 and pneumonia diseases using data mining and transfer learning algorithms with focal loss from chest X-ray images

Pneumonia is a lung infection that produces severe inflammation, which can be caused by viruses, bacteria, or fungi. The global occurrence of pneumonia cases has been notably affected by the outbreak of Coronavirus Disease 2019 (COVID-19). Recent studies have demonstrated the capability of Deep Learning (DL) algorithms in detecting COVID-19 and pneumonia in medical images like X-rays. In this study, we employ four different Chest X-ray (CXR) datasets to evaluate four pre-trained Convolutional Neural Network (CNN) models: InceptionResNet V2, Inception V3, Xception, and MobileNet. These models were fine-tuned using either the Cross-Entropy (CE) loss function only for balanced datasets or separately using both the CE and Focal Loss (FL) functions for imbalanced datasets. For the first dataset, the InceptionResNet V2 model achieved the highest multi-classification accuracy of 88.63%. The superior model for the second and fourth datasets is Inception V3, with 94.35% and 97.67% multi-classification accuracy, respectively. For the third dataset, the best model is Xception, with a binary classification accuracy of 100.00%. Our results emphasize the significance of using the FL function in solving class imbalance problems in DL models. Additionally, it highlights the effectiveness of individual DL models in detecting different pneumonia infections using different CXR datasets.

Bacterial Disease Detection of Cherry Plant Using Deep Features

Although the cherry plant is widely grown in the world and Turkey, it is a fruit tree that is difficult to grow and maintain. It can be exposed to various pesticide diseases, especially during fruiting. Today, approaches based on expert reviews and analyses are used for the identification of these diseases. In addition, cherry producers are trying to detect diseases with their knowledge based on experience. Computer-aided agricultural analysis systems are also being developed depending on the rapid developments in technology. These systems help to monitor all processes from planting, cultivation, and harvesting of agricultural products and to make decisions to grow the products healthily. One of the most important issues to be detected and monitored with these systems is plant diseases. The features of the cherry plant disease will be determined by using a pre-trained convolutional neural network (CNN) model which is DarkNet-19, within the scope of this study. These machine learning-based features have been used for the detection of bacteria-based diseases commonly seen on the leaves of cherry plants. The acquired features are classified with Linear Discriminant Analysis, K-Nearest Neighbor, and Support Vector Machine classifiers to solve the multi-class problem including diseased (less and very) and healthy plants. The experimental results show that a success rate of 88.1% was obtained in the detection of the disease.

Prediction of Obliteration After the Gamma Knife Radiosurgery of Arteriovenous Malformations Using Hand-Crafted Radiomics and Deep-Learning Methods.

Brain arteriovenous malformations (bAVMs) are vascular abnormalities that can be treated with embolization or radiotherapy to prevent the risk of future rupture. In this study, we use hand-crafted radiomics and deep learning techniques to predict favorable vs. unfavorable outcomes following Gamma Knife radiosurgery (GKRS) of bAVMs and compare their prediction performances. One hundred twenty-six patients seen at one academic medical center for GKRS obliteration of bAVMs over 15 years were retrospectively reviewed. Forty-two patients met the inclusion criteria. Favorable outcomes were defined as complete nidus obliteration demonstrated on cerebral angiogram and asymptomatic recovery. Unfavorable outcomes were defined as incomplete obliteration or complications relating to the AVM that developed after GKRS. Outcome predictions were made using a random forest model with hand-crafted radiomic features and a fine-tuned ResNet-34 convolutional neural network (CNN) model. The performance was evaluated by using a ten-fold cross-validation technique. The average accuracy and area-under-curve (AUC) values of the Random Forest Classifier (RFC) with radiomics features were 68.5 ±9.80% and 0.705 ±0.086, whereas those of the ResNet-34 model were 60.0 ±11.9% and 0.694 ±0.124. Four radiomics features used with RFC discriminated unfavorable response cases from favorable response cases with statistical significance. When cropped images were used with ResNet-34, the accuracy and AUC decreased to 59.3 ± 14.2% and 55.4 ±10.4%, respectively. A hand-crafted radiomics model and a pre-trained CNN model can be fine-tuned on pre-treatment MRI scans to predict clinical outcomes of AVM patients undergoing GKRS with equivalent prediction performance. The outcome predictions are promising but require further external validation on more patients.

Sign Language Recognition Using Convolutional Neural Network

This project introduces a real-time sign language detection system powered by Convolutional Neural Networks (CNNs), designed to aid individuals with hearing impairments in communication. Leveraging OpenCV for video capture and hand detection, along with custom modules for hand tracking and image classification, the system seamlessly integrates deep learning methodologies. A pre-trained CNN model, trained on a comprehensive dataset of sign language gestures, forms the core of the system, ensuring accurate classification in real-time. By capturing video frames from a webcam, detecting hands, and processing them through the CNN model, the system provides immediate feedback by overlaying predicted gestures onto the video stream. Through its effective implementation, this project underscores the potential of CNNs in facilitating accessibility and inclusivity, while also paving the way for future enhancements and applications in assistive technology and human-computer interaction. Top of Form Keywords— Real-time, Sign Language Detection, Convolutional Neural Networks (CNNs), Communication, Hearing Impairments, Deep Learning.

Deep learning for discrimination of active and inactive lesions in multiple sclerosis using non-contrast FLAIR MRI: A multicenter study

BackgroundWithin the domain of multiple sclerosis (MS), the precise discrimination between active and inactive lesions bears immense significance. Active lesions are enhanced on T1-weighted MRI images after administration of gadolinium-based contrast agents, which brings about associated complexities. This study investigates the potential of deep learning to differentiate between active and inactive lesions in MS using non-contrast FLAIR-type MRI data, presenting a non-invasive alternative to conventional gadolinium-based MRI methods. MethodsThe dataset encompasses 9097 lesion images collected from 130 MS patients across four distinct imaging centers, with post-contrast T1-weighted images as the benchmark reference. We initially identified and labeled the lesions and carefully selected corresponding regions of interest (ROIs). These ROIs were employed as inputs for a convolutional neural network (CNN) to predict lesion status. Also, transfer learning was utilized, incorporating 12 pre-trained CNN models. Subsequently, an ensemble technique was applied to 3 of best models, followed by a systematic comparison of the results. ResultsThrough a 5-fold cross-validation, our custom designed network exhibited an average accuracy of 85 %, a sensitivity of 95 %, a specificity of 75 %, and an AUC value of 0.90. Among the pre-trained models, ResNet50 emerged as the most effective, achieving a specificity of 58 %, an accuracy of 75 %, a sensitivity of 91 %, and an AUC value of 0.81. Our comprehensive evaluations encompassed the receiver operating characteristic curve, precision-recall curve, and confusion matrix analyses. ConclusionThe findings underscore the efficacy of the proposed CNN, trained on FLAIR MRI data ROIs, in accurately discerning active and inactive lesions without reliance on contrast agents. Our multicenter study of 130 patients with diverse imaging devices outperforms the other single-center studies, achieving superior sensitivity and specificity. Unlike studies using multiple modalities, our exclusive use of FLAIR images streamlines the process, and our streamlined approach, excluding conventional pre-processing, demonstrates efficiency. The external validation conducted on diverse datasets, coupled with the analysis of dilated masks, underscores the adaptability and efficacy of our custom CNN model in discerning between active and inactive lesions.

COVID-19 Hierarchical Classification Using a Deep Learning Multi-Modal.

Coronavirus disease 2019 (COVID-19), originating in China, has rapidly spread worldwide. Physicians must examine infected patients and make timely decisions to isolate them. However, completing these processes is difficult due to limited time and availability of expert radiologists, as well as limitations of the reverse-transcription polymerase chain reaction (RT-PCR) method. Deep learning, a sophisticated machine learning technique, leverages radiological imaging modalities for disease diagnosis and image classification tasks. Previous research on COVID-19 classification has encountered several limitations, including binary classification methods, single-feature modalities, small public datasets, and reliance on CT diagnostic processes. Additionally, studies have often utilized a flat structure, disregarding the hierarchical structure of pneumonia classification. This study aims to overcome these limitations by identifying pneumonia caused by COVID-19, distinguishing it from other types of pneumonia and healthy lungs using chest X-ray (CXR) images and related tabular medical data, and demonstrate the value of incorporating tabular medical data in achieving more accurate diagnoses. Resnet-based and VGG-based pre-trained convolutional neural network (CNN) models were employed to extract features, which were then combined using early fusion for the classification of eight distinct classes. We leveraged the hierarchal structure of pneumonia classification within our approach to achieve improved classification outcomes. Since an imbalanced dataset is common in this field, a variety of versions of generative adversarial networks (GANs) were used to generate synthetic data. The proposed approach tested in our private datasets of 4523 patients achieved a macro-avg F1-score of 95.9% and an F1-score of 87.5% for COVID-19 identification using a Resnet-based structure. In conclusion, in this study, we were able to create an accurate deep learning multi-modal to diagnose COVID-19 and differentiate it from other kinds of pneumonia and normal lungs, which will enhance the radiological diagnostic process.

Ultrasound-based deep learning radiomics model for differentiating benign, borderline, and malignant ovarian tumours: a multi-class classification exploratory study.

Accurate preoperative identification of ovarian tumour subtypes is imperative for patients as it enables physicians to custom-tailor precise and individualized management strategies. So, we have developed an ultrasound (US)-based multiclass prediction algorithm for differentiating between benign, borderline, and malignant ovarian tumours. We randomised data from 849 patients with ovarian tumours into training and testing sets in a ratio of 8:2. The regions of interest on the US images were segmented and handcrafted radiomics features were extracted and screened. We applied the one-versus-rest method in multiclass classification. We inputted the best features into machine learning (ML) models and constructed a radiomic signature (Rad_Sig). US images of the maximum trimmed ovarian tumour sections were inputted into a pre-trained convolutional neural network (CNN) model. After internal enhancement and complex algorithms, each sample's predicted probability, known as the deep transfer learning signature (DTL_Sig), was generated. Clinical baseline data were analysed. Statistically significant clinical parameters and US semantic features in the training set were used to construct clinical signatures (Clinic_Sig). The prediction results of Rad_Sig, DTL_Sig, and Clinic_Sig for each sample were fused as new feature sets, to build the combined model, namely, the deep learning radiomic signature (DLR_Sig). We used the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) to estimate the performance of the multiclass classification model. The training set included 440 benign, 44 borderline, and 196 malignant ovarian tumours. The testing set included 109 benign, 11 borderline, and 49 malignant ovarian tumours. DLR_Sig three-class prediction model had the best overall and class-specific classification performance, with micro- and macro-average AUC of 0.90 and 0.84, respectively, on the testing set. Categories of identification AUC were 0.84, 0.85, and 0.83 for benign, borderline, and malignant ovarian tumours, respectively. In the confusion matrix, the classifier models of Clinic_Sig and Rad_Sig could not recognise borderline ovarian tumours. However, the proportions of borderline and malignant ovarian tumours identified by DLR_Sig were the highest at 54.55% and 63.27%, respectively. The three-class prediction model of US-based DLR_Sig can discriminate between benign, borderline, and malignant ovarian tumours. Therefore, it may guide clinicians in determining the differential management of patients with ovarian tumours.

A Fuzzy Minkowski Distance-based fusion of convolutional neural networks for gastrointestinal disease detection

Accurate classification of gastrointestinal (GI) conditions from medical images is a critical task for facilitating timely diagnosis and effective treatment. However, the reliance on manual diagnosis introduces the possibility of human errors. In response, researchers have been tirelessly working to develop robust computerized methods that can significantly enhance diagnostic accuracy. In this study, we present a novel approach designed to elevate the precision of classification tasks through the utilization of a Fuzzy Minkowski Distance-based Ensemble Model. Our ensemble model is accurately constructed by integrating three well-established pre-trained convolutional neural network (CNN) models: MobileNet, ResNet101V2, and Xception. To enhance the robustness of these base models, we incorporate ResNeXt block, effectively amplifying their ability in feature extraction and representation. Furthermore, we extract probabilities from these models and aggregate them using a fuzzy Minkowski Distance approach. This technique serves to minimize error values between observed and ground-truth data, leading to a further enhancement in detection accuracy. For our experiments, we utilize a publicly available dataset containing 6000 endoscopy images. These images are not only enhanced through the application of image contrast enhancement techniques but are also subjected to augmentation processes, effectively enhancing the dataset's diversity and contributing to improved performance in the classification task. After conducting comprehensive assessments, we highlight the strength of our ensemble model compared to single base models and existing methods. The ensemble consistently achieves an impressive accuracy of 99.62%, showcasing its potential as a powerful tool for accurate diagnosis in the field of gastroenterology.

A deep learning framework for accurate diagnosis of colorectal cancer using histological images

Colorectal cancer (CRC) is one of the most prevalent malignancies worldwide, with high mortality and incidence rates. Early detection of the disease may increase the probability of survival, making it critical to develop effective procedures for precise treatment. In the past few years, there has been an increased use of deep learning techniques in image classification that aid in the detection of various types of cancer. In this study, convolutional neural network (CNN) models were used to classify colorectal cancer into benign and malignant. After applying various data preprocessing techniques to the image dataset, we evaluated our prototypes using three distinct subsets of testing data, representing 20%, 30%, and 40% of the total dataset. Additionally, four pre-trained CNN models (ResNet-18, ResNet-50, GoogLeNet, and MobileNetV2) were trained, and the network architectural techniques were compared by applying the Adam optimizer. Finally, we assessed the performance of algorithms in terms of accuracy, sensitivity, specificity, precision, F1-score, and area under the receiver operating characteristic (ROC) curve (AUC). In this research, deep learning approaches demonstrated high efficacy in accurately diagnosing colorectal cancer. This indicates that these techniques have an important and significant value for advancing medical research.

Abstract 2329: Cancer region definition using spatial gene expression patterns by super resolution reconstruction algorithm for spatial transcriptomics data

Abstract Background: Spatially resolved transcriptomics (ST) has enabled a variety of cancer research on the heterogeneity of tumor microenvironment. However, when identifying cancer boundaries based on ST, the limited resolution of ST such as Visium hinders the accurate identification of cancerous regions. To address these issues, we have developed a novel approach for accurately classifying regions of the tumor by using deep image prior (DIP) algorithm to convert Visium data into high-resolution images. Methods: We proposed SuperST, which utilizes deep image prior to transform low-resolution ST data into high-resolution images, providing more accurate spatial marker expression patterns. This method was used to define cancer regions by combining with a pre-trained deep learning algorithm that recognizes spatial patterns. ST data of hepatocellular carcinoma (HCC) patients were used to apply SuperST. Firstly, 400 spatially variable genes were identified. SuperST leverages the deep image prior (DIP) algorithm and a U-Net neural network to transform ST data into high-resolution ones without needing ground-truth high-resolution data. The final high-resolution images of gene expression data were used to extract 512-dimensional feature vectors for each gene via a pre-trained convolutional neural network model (VGG16). Subsequently, we apply K-means clustering to these features to identify transcriptionally distinct tissue regions considering spatial information and compared with the human-labeled cancer regions. Results: The cancer region detection using spatial gene expression data and our algorithm yielded results that substantially align with the human annotations on the H&E images. Furthermore, compared to the cancer definition obtained using the CopyKat and SPACET algorithms, our results were more spatially consistent for cancerous regions as well as providing high-resolution image-level definition of cancer region margin. This indicates that our algorithm better utilizes spatial information and more effectively overcomes the low resolution of ST data. Conclusion: This research demonstrates the effective application of our SuperST algorithm to the challenging problem of cancer region detection only using spatial gene expression data. It suggests that our algorithm could be utilized for a variety of problems in spatial biology in cancer research. Citation Format: Jeongbin Park, Seungho Cook, Dongjoo Lee, Jinyeong Choi, Seongjin Yoo, Hyung-Jun Im, Daeseung Lee, Hongyoon Choi. Cancer region definition using spatial gene expression patterns by super resolution reconstruction algorithm for spatial transcriptomics data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2329.

Abstract 4938: Prediction of gene mutation from colorectal adenocarcinoma whole slide images via integrated deep learning pipeline

Abstract Colorectal cancer (CRC) stands as one of the deadliest cancer types worldwide. Identifying specific gene mutations crucial for targeted therapy is pivotal for CRC treatment. However, the current genetic testing method involves liquid biopsy, which is often cumbersome and cost-intensive. Histopathological whole-slide images (WSIs) are routinely generated for diagnostic purposes and can offer valuable insights into cellular heterogeneity. In this study, we aim to develop an integrated deep learning pipeline for predicting TP53 mutations in CRC, one of the most frequently mutated genes in CRC. We commenced by collecting 424 diagnostic WSIs from the TCGA-COAD cohorts. Each WSI was split as small patches at a resolution of 0.5 micrometers per pixel for following analyses. Quality control was implemented using Otsu thresholding and Gaussian blur filtering, both at the slide-level masking and patch-level filtering stages. Additionally, we applied the Macenko algorithm for color normalization to eliminate systematic errors. Following image preprocessing, we constructed a tissue-type classification model based on VGG19, which was trained and tested using the NCT-CRC-HE-100K and CRC-VAL-HE-7K datasets, both containing pixels with tissue types labeled by human experts. The trained classifier was then used to identify adenocarcinoma epithelium patches from TCGA-COAD. Subsequently, we applied multiple pre-trained Convolutional Neural Network (CNN) models, including ResNet-101, DenseNet-201, and Inception-ResNet-v2, to predict TP53 mutations for the identified adenocarcinoma epithelium patches from TCGA-COAD. A novel integration approach was employed to aggregate patch-level predictions into slide-level outcomes. We evaluated the trained models using various metrics, including accuracy, precision, recall, F1 score, AUC (Area Under the Curve), and Cohen's Kappa coefficient. The tissue-type classification model exhibited robust performance, achieving an overall accuracy of 94.7%, a Cohen's Kappa coefficient of 94%, a precision of 95.3%, a recall of 99.76%, and an F1 score of 97.48% for identifying adenocarcinoma epithelium patches. Regarding the TP53 mutation prediction models, ResNet-101, DenseNet-201, and Inception-ResNet-v2 exhibit the predictive ability to both patch level (AUC=65%) and slide level (AUC=70%). In summary, the proposed deep learning pipeline offers an efficient way to acquire gene mutation information from WSIs. Moreover, it demonstrates a capability to discover potential molecular aberration from the clinical images. Citation Format: Geng-Yun Tien, Liang-Chuan Lai, Tzu-Pin Lu, Mong-Hsun Tsai, Hsiang-Han Chen, Eric Y. Chuang. Prediction of gene mutation from colorectal adenocarcinoma whole slide images via integrated deep learning pipeline [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4938.