Pre-trained ResNet Research Articles

Comparative Analysis of Neural Network Architectures for Automated Fracture Detection in Hand X-ray Images

The application of several neural network architectures—including Fully Connected Networks (FCN), Convolutional Neural Networks (CNN), pretrained ResNet, Vision Transformer (ViT-B-16)—for the classification of hand X-ray images into "Fractured" and "Not Fractured"—categories is investigated in this work. The main goals are to evaluate these models' fracture detection ability and determine which architectural design fits this work. Because transfer learning let the model use past information from big-scale picture datasets, the pretrained ResNet model emerged as the most effective with high accuracy, stability, and resilience. The bespoke CNN also performed well, displaying excellent feature extraction powers especially for medical imaging. But the non-pretrained ResNet model overfitted, meaning deeper networks find it difficult to generalize without pretraining. Though innovative, the Vision Transformer performed poorly since it depends on a lot of training data and finds difficult learning of intricate spatial properties from little datasets. Although acting as a baseline, the FCN's simple architecture and incapacity to detect spatial hierarchies in images meant it could not match the efficacy of CNN models. Emphasizing the important function of transfer learning in clinical applications, the results show that pretrained CNN architectures, especially ResNet, offer the most consistent and accurate method for automatic fracture diagnosis in medical pictures.

DynaSeg: A deep dynamic fusion method for unsupervised image segmentation incorporating feature similarity and spatial continuity

Our work tackles the fundamental challenge of image segmentation in computer vision, which is crucial for diverse applications. While supervised methods demonstrate proficiency, their reliance on extensive pixel-level annotations limits scalability. We introduce DynaSeg, an innovative unsupervised image segmentation approach that overcomes the challenge of balancing feature similarity and spatial continuity without relying on extensive hyperparameter tuning. Unlike traditional methods, DynaSeg employs a dynamic weighting scheme that automates parameter tuning, adapts flexibly to image characteristics, and facilitates easy integration with other segmentation networks. By incorporating a Silhouette Score Phase, DynaSeg prevents undersegmentation failures where the number of predicted clusters might converge to one. DynaSeg uses CNN-based and pre-trained ResNet feature extraction, making it computationally efficient and more straightforward than other complex models. Experimental results showcase state-of-the-art performance, achieving a 12.2% and 14.12% mIOU improvement over current unsupervised segmentation approaches on COCO-All and COCO-Stuff datasets, respectively. We provide qualitative and quantitative results on five benchmark datasets, demonstrating the efficacy of the proposed approach. Code available at \\url{https://github.com/RyersonMultimediaLab/DynaSeg}

Few-Shot Conditional Learning: Automatic and Reliable Device Classification for Medical Test Equipment.

The limited availability of specialized image databases (particularly in hospitals, where tools vary between providers) makes it difficult to train deep learning models. This paper presents a few-shot learning methodology that uses a pre-trained ResNet integrated with an encoder as a backbone to encode conditional shape information for the classification of neonatal resuscitation equipment from less than 100 natural images. The model is also strengthened by incorporating a reliability score, which enriches the prediction with an estimation of classification reliability. The model, whose performance is cross-validated, reached a median accuracy performance of over 99% (and a lower limit of 73.4% for the least accurate model/fold) using only 87 meta-training images. During the test phase on complex natural images, performance was slightly degraded due to a sub-optimal segmentation strategy (FastSAM) required to maintain the real-time inference phase (median accuracy 87.25%). This methodology proves to be excellent for applying complex classification models to contexts (such as neonatal resuscitation) that are not available in public databases. Improvements to the automatic segmentation strategy prior to the extraction of conditional information will allow a natural application in simulation and hospital settings.

Automated Retinal Image Analysis to Detect Optic Nerve Hypoplasia

Identification of the optic disc and fovea is crucial for automating the diagnosis and screening of retinal diseases. Based on quantitative calculations, this study presents a decision support system for doctors that automatically detect optic nerve hypoplasia. For disease diagnosis, U-Net architecture is used, which uses a pre-trained ResNet encoder to segment the optic disc and fovea structures. An important aspect of the proposed technique is that pretrained ResNet and U-Net are used together, providing robust performance in the detection of optic nerve hypoplasia. Our proposed architecture was tested on retinal images from Messidor, Diaretdb1, DRIVE, HRF, APTOS, and IDRID. In addition, a special database called ONH-NET was created based on 189 retinal images obtained from Düzce University, Department of Ophthalmology. Messidor database test images showed,0. 9069 IOU Score, 0.9626 Sensitivity, 0.9411 Precision, 0.9974 Accuracy and 0.9505 dice-coefficient values in optic disc detection, and 0.8282 IOU score, 0.8442 sensitivity, 0.8252 precision, 0.8992 Accuracy, 0.7873 dice coefficient values were obtained in fovea detection. We computed diameter optic disc to macula radius ratios from segmented optic disc and fovea for screening optic nerve hypoplasia and achieved 100% success.

Prediction of melanoma metastasis using dermatoscopy deep features: An international multicenter cohort study.

9565 Background: Machine learning algorithms, and especially convolutional neural networks, demonstrated modest accuracy on the prediction of melanoma metastasis, based on histological images and clinicopathological information. Whether dermatoscopy deep features could serve as biomarker for the prediction of melanoma metastasis, remains an underexplored area in medical research. Methods: An international, multicenter, cohort study of cutaneous melanoma patients from 3 different continents was conducted. Patients with cutaneous melanoma, who had available clinical and dermatoscopic images and an adequate follow-up time for the development of metastasis (both locoregional and distant) were included. We utilized a support vector machine (SVM) classifier, to distinguish between melanomas that metastasized and those that did not. We used a pre-trained ResNet 50 network, we separated dataset into training set and testing set, stratified by TNM-stage, and to ensure robustness and guard against biased data selection, the stratified split into training and testing sets was repeated five times, resulting in five different training-test sets. The primary outcome was the comparison of the prognostic performance of deep dermatoscopy features based on SVM (model 1) to the performance of established prognostic factors of melanoma, such as Breslow thickness and ulceration (model 2) and to a combined model using deep features and histopathologic factors (model 3). A secondary aim was to examine the diagnostic performance of model 1 in stage IIB and IIC patients at diagnosis. The prognostic performance was assessed using the Area Under the Curve (AUC) and the True Positive Rate (TPR) at a True Negative Rate at 70%. Results: 712 patients were included, 465 (65.3%) non-metastatic and 247 (34.7%) metastatic, within a median follow-up of 60 months. The SVM model demonstrated mean AUC 0.84 (95% CI 0.80 – 0.87) and TPR 0.81 (95%CI 0.73 – 0.90). Similar results were shown for model 2 and model 3, and no statistically significant differences among models were detected in terms of AUC (De Long’s test, p>0.05 and ANOVA Kruskal-Wallis p>0.05). Regarding IIB/IIC patients and combining data from five test sets, SVM correctly classified as metastatic 21 out of 23 (91.3%) stage IIB and 21 out of 23 (91.3%) stage IIC patients, who eventually developed metastasis during follow-up. Conclusions: Our findings suggest that dermatoscopy deep features could offer an immediate, in vivo prediction of melanoma metastasis prior to excision. This advancement holds significant clinical importance, prioritizing high-risk patients for neoadjuvant treatment or guiding selection of patients who might benefit from adjuvant therapy.

Implementation of satelite image classification using PyTorch

This article presents a practical implementation of the satellite image classification task using the PyTorch deep learning library. The aim of the work is to develop a neural network model capable of classifying satellite images into four classes: water, sands, clouds, and greenery. The model is trained using the pre-trained ResNet architecture. The proposed approach allows for automatic classification of satellite images, fa-cilitating the analysis of large amounts of data, and has been tested on Sentinel-2 satellite images acquired from March to August 2023 for parts of Kherson and Mykolaiv regions, Ukraine, and achieved an average accuracy of 95%. The presented research demonstrates the practical use of modern deep learning methods, in particular the PyTorch library, to solve actual problems of satellite image analysis and classification.

EDUVQA – Visual Question Answering: An Educational Perspective

Increasing applications of artificial intelligence in the field of education have changed the way school children learn various concepts. Educational Visual Question Answering or EDUVQA is one such application that allows students to interact directly with images, ask educational questions, and get the correct answer. Two major challenges faced by educational VQA are the lack of availability of domain-specific datasets and often it requires referring to the external knowledge bases to answer open-domain questions. We propose a novel EDUVQA model developed especially for educational purposes and introduce our own EDUVQA dataset. The dataset consists of four categories of images - animals, plants, fruits, and vegetables. The majority of the currently used techniques focus on the extraction of picture and question characteristics in order to discover the joint feature embeddings via multimodal fusion or attention mechanisms. We propose a different method that aims to better utilize the semantic knowledge present in images. Our approach entails building an EDUVQA dataset using educational images, where each data point is made up of an image, a question that corresponds to it, a valid response, and a fact that supports it. The fact is created in the form of <S,V,O> triplet where ‘s’ denotes a subject, ‘v’ a verb, and ‘o’ an object. First, an SVO detector model is trained on EDUVQA dataset capable of predicting the Subject, Verb, and Object present in the image-question pair. Using this <S,V,O> triplet, the most relevant facts from our fact base are extracted. The final answer is predicted using these extracted facts, image, and question attributes. The image features are extricated using pretrained ResNet and question features using a pre-trained BERT model. We have optimized and improved on the current methodologies that use a Relation-based approach and built our SVO-detector model that outperforms current models by 10%.

Privacy-Preserving Breast Cancer Classification: A Federated Transfer Learning Approach.

Breast cancer is deadly cancer causing a considerable number of fatalities among women in worldwide. To enhance patient outcomes as well as survival rates, early and accurate detection is crucial. Machine learning techniques, particularly deep learning, have demonstrated impressive success in various image recognition tasks, including breast cancer classification. However, the reliance on large labeled datasets poses challenges in the medical domain due to privacy issues and data silos. This study proposes a novel transfer learning approach integrated into a federated learning framework to solve the limitations of limited labeled data and data privacy in collaborative healthcare settings. For breast cancer classification, the mammography and MRO images were gathered from three different medical centers. Federated learning, an emerging privacy-preserving paradigm, empowers multiple medical institutions to jointly train the global model while maintaining data decentralization. Our proposed methodology capitalizes on the power of pre-trained ResNet, a deep neural network architecture, as a feature extractor. By fine-tuning the higher layers of ResNet using breast cancer datasets from diverse medical centers, we enable the model to learn specialized features relevant to different domains while leveraging the comprehensive image representations acquired from large-scale datasets like ImageNet. To overcome domain shift challenges caused by variations in data distributions across medical centers, we introduce domain adversarial training. The model learns to minimize the domain discrepancy while maximizing classification accuracy, facilitating the acquisition of domain-invariant features. We conducted extensive experiments on diverse breast cancer datasets obtained from multiple medical centers. Comparative analysis was performed to evaluate the proposed approach against traditional standalone training and federated learning without domain adaptation. When compared with traditional models, our proposed model showed a classification accuracy of 98.8% and a computational time of 12.22s. The results showcase promising enhancements in classification accuracy and model generalization, underscoring the potential of our method in improving breast cancer classification performance while upholding data privacy in a federated healthcare environment.

Deep Learning for Identifying Iran’s Cultural Heritage Buildings in Need of Conservation Using Image Classification and Grad-CAM

The cultural heritage buildings (CHB), which are part of mankind’s history and identity, are in constant danger of damage, or in extreme cases, complete destruction. Thus, it’s of utmost importance to preserve them by identifying the existent, or presumptive, defects using novel methods so that renovation processes can be done in a timely manner and with higher accuracy. The main goal of this research is to use new Deep Learning (DL) methods in the process of preserving CHBs (situated in Iran); a goal that has been neglected especially in developing countries such as Iran, as these countries still preserve their CHBs using manual, and even archaic, methods that need direct human supervision. Having proven their effectiveness and performance when it comes to processing images, the Convolutional Neural Networks (CNNs) are a staple in computer vision (CV) literacy and this article is not exempt. When lacking enough CHB images, training a CNN from scratch would be very difficult and prone to overfitting; that’s why we opted to use a technique called transfer learning (TL) in which we used pre-trained ResNet, MobileNet, and Inception networks, for classification. Even more, the Grad-CAM was utilized to localize the defects to some extent. The final results were very favorable, compared to similar articles. We reached 94% in Precision, Recall, and F1-Score with our fine-tuned MobileNetV2 model, which showed a 4%–5% improvement over other similar works. The final proposed model can pave the way for moving from manual to unmanned CHB conservation, hence an increase in accuracy and a decrease in human-induced errors.

Contextual feature representation for image-based insider threat classification

With the rapid development of technology, insider threat incidents frequently occur in organizations. Detecting insider threats is an essential task in network infrastructure security. In this paper, we design an attention module to extract contextual features and augment abnormal features to generate high-quality images representing user behavior. Then, we use pre-trained ResNet and multi-source feature fusion on behavioral, psychological, and role features, intending to identify malicious insiders accurately. The proposed approaches are evaluated using the CMU-CERT Insider Threat Dataset. Experimental results show the effectiveness of methods and outperform other state-of-the-art methods.

Motion-aware object tracking for aerial images with deep features and discriminative correlation filter

Object tracking is a challenging task which is required for different problems such as surveillance, traffic analysis and human-computer interaction. The problem of tracking an object can be considered in different categories such as single object tracking, multiple object tracking, short-term tracking, long-term tracking, tracking by detection and detection-free tracking. This study focuses on detection-free tracking for ground targets on aerial images. The studies in the literature show that correlation filter and deep learning based object trackers perform well recently. This paper proposes a new correlation filter-based tracker containing a strategy for re-detection issue. We improve the performance of correlation filter-based tracker by adding a lightweight re-detection ability to the correlation filter tracker in case of a long occlusion or complete loss of target. We use deep features to train Discriminative Correlation Filter(DCF) by integrating sub-networks from pre-trained ResNet and SAND models. The experimental results on the popular UAV123L dataset show that the proposed method(MADCF) improves the performance of DCF tracker and have a reasonable performance for long-term tracking problem. Moreover, we prepare a new tracking dataset (PESMOD tracking) consisting of UAV images, and we evaluate the proposed method and state-of-the-art method in this dataset. We observed that the proposed method performs much better in ground target tracking from VIVID and PESMOD aerial datasets. The proposed MADCF tracker performs better for small targets tracked by UAVs compared to the deep learning-based trackers. The source code and prepared dataset are available at http://github.com/mribrahim/MADCF

Duplex-Hierarchy Representation Learning for Remote Sensing Image Classification.

Remote sensing image classification (RSIC) is designed to assign specific semantic labels to aerial images, which is significant and fundamental in many applications. In recent years, substantial work has been conducted on RSIC with the help of deep learning models. Even though these models have greatly enhanced the performance of RSIC, the issues of diversity in the same class and similarity between different classes in remote sensing images remain huge challenges for RSIC. To solve these problems, a duplex-hierarchy representation learning (DHRL) method is proposed. The proposed DHRL method aims to explore duplex-hierarchy spaces, including a common space and a label space, to learn discriminative representations for RSIC. The proposed DHRL method consists of three main steps: First, paired images are fed to a pretrained ResNet network for extracting the corresponding features. Second, the extracted features are further explored and mapped into a common space for reducing the intra-class scatter and enlarging the inter-class separation. Third, the obtained representations are used to predict the categories of the input images, and the discrimination loss in the label space is minimized to further promote the learning of discriminative representations. Meanwhile, a confusion score is computed and added to the classification loss for guiding the discriminative representation learning via backpropagation. The comprehensive experimental results show that the proposed method is superior to the existing state-of-the-art methods on two challenging remote sensing image scene datasets, demonstrating that the proposed method is significantly effective.

Quantitative analysis of [formula omitted] thin film micrographs with machine learning

Isolating the features associated with different materials growth conditions is important to facilitate the tuning of these conditions for effective materials growth and characterization. This study presents machine learning models for classifying atomic force microscopy (AFM) images of thin film MoS2 based on their growth temperatures. By employing nine different algorithms and leveraging transfer learning through a pretrained ResNet model, we identify an effective approach for accurately discerning the characteristics related to growth temperature within the AFM micrographs. Robust models with test accuracies of up to 70% were obtained, with the best performing algorithm being an end-to-end ResNet fine-tuned on our image domain. Class activation maps and occlusion attribution reveal that crystal quality and domain boundaries play crucial roles in classification, with models exhibiting the ability to identify latent features that humans could potentially miss. Overall, the models demonstrated high accuracy in identifying thin films grown at different temperatures despite limited and imbalanced training data as well as variation in growth parameters besides temperature, showing that our models and training protocols are suitable for this and similar predictive tasks for accelerated 2D materials characterization.

Rule-based deep learning method for prognosis of neonatal hypoxic-ischemic encephalopathy by using susceptibility weighted image analysis.

Susceptibility weighted imaging (SWI) of neonatal hypoxic-ischemic brain injury can provide assistance in the prognosis of neonatal hypoxic-ischemic encephalopathy (HIE). We propose a convolutional neural network model to classify SWI images with HIE. Due to the lack of a large dataset, transfer learning method with fine-tuning a pre-trained ResNet 50 is introduced. We randomly select 11 datasets from patients with normal neurology outcomes (n = 31) and patients with abnormal neurology outcomes (n = 11) at 24 months of age to avoid bias in classification due to any imbalance in the data. We develop a rule-based system to improve the classification performance, with an accuracy of 0.93 ± 0.09. We also compute heatmaps produced by the Grad-CAM technique to analyze which areas of SWI images contributed more to the classification patients with abnormal neurology outcome. Such regions that are important in the classification accuracy can interpret the relationship between the brain regions affected by hypoxic-ischemic and neurodevelopmental outcomes of infants with HIE at the age of 2 years.

Image segmentation of impacted mesiodens using deep learning.

This study aimed to evaluate the performance of deep learning algorithms for the classification and segmentation of impacted mesiodens in pediatric panoramic radiographs. A total of 850 panoramic radiographs of pediatric patients (aged 3-9 years) was included in this study. The U-Net semantic segmentation algorithm was applied for the detection and segmentation of mesiodens in the upper anterior region. For enhancement of the algorithm, pre-trained ResNet models were applied to the encoding path. The segmentation performance of the algorithm was tested using the Jaccard index and Dice coefficient. The diagnostic accuracy, precision, recall, F1-score and time to diagnosis of the algorithms were compared with those of human expert groups using the test dataset. Cohen's kappa statistics were compared between the model and human groups. The segmentation model exhibited a high Jaccard index and Dice coefficient (>90%). In mesiodens diagnosis, the trained model achieved 91-92% accuracy and a 94-95% F1-score, which were comparable with human expert group results (96%). The diagnostic duration of the deep learning model was 7.5 seconds, which was significantly faster in mesiodens detection compared to human groups. The agreement between the deep learning model and human experts is moderate (Cohen's kappa = 0.767). The proposed deep learning algorithm showed good segmentation performance and approached the performance of human experts in the diagnosis of mesiodens, with a significantly faster diagnosis time.

#319 : Blastocyst Formation Prediction Based on Deep Learning Model from 3-Day Embryo Images in Time-Lapse Incubator Using Data Augmentation

Background and Aims: Current literature suggests that blastocyst ET (Embryo Transfer) at day 5 improves pregnancy outcomes compared with cleavage ET at day 3. However, blastocyst ET poses potential challenges due to the risk of developmental arrest at the cleavage stage. Therefore, accurately predicting blastocyst formation in embryos will help determining optimal days for embryo culture. The aim of our study is to develop a deep learning-based classification model that can predict blastocyst formation based on 3-day embryo images captured from Time-Lapse incubators. Method: A total of 200 embryo images were collected at 72 hours after ICSI (Intracytoplasmic Sperm Injection), with 100 forming blastocysts and 100 failing to do so. The images were annotated by RectLabel for object detection and segmentation, and data augmentation techniques were applied to enhance the dataset. A pre-trained ResNet model was used as the basis for our classifier model, and optimization was performed using the Adam optimizer and BCEWithLogitsLoss as the loss function. Model performance was evaluated through classification accuracy and AUC score. Results: The CNN-based classifier models demonstrated a considerable accuracy of 87.5% and an AUC of 0.896 in predicting blastocyst formation, with sensitivity and specificity of 91.7% and 81.3%, respectively. It was found that without data augmentation, the model degraded to an accuracy of 62.5% and an AUC of 0.631. Conclusion: Our study competently developed a classification model for predicting blastocyst formation using 3-day embryo images from a Time-Lapse incubator and demonstrated the effectiveness of data augmentation techniques in limited data sets. This deep learning-based model may contribute as a beneficial tool in IVF (In Vitro Fertilization) process.

ResNet50: Automated Fabric Defect Detection and Classification based on a Deep Learning Approach

"The demand for an intelligent computer-based system for visual inspection is on the rise in the textile business, especially among those who place a premium on textile quality. In this research, we present a fully automated AI-driven algorithm for defect detection in fabric, one that makes use of deep neural network models that have already been pre-trained. In order to train the networks, the fabric images go through a series of pre-processing steps where typical image processing techniques are applied. In order to train and categorize various fabric flaws, we use Deep Convolutional Neural Networks (DCNNs) and the pre-trained ResNeT network. The system obtains a best classification accuracy of 96.40% in simulations utilizing preexisting textile datasets. This model's detection and classification system can help human operators spot flaws in the fabric manufacturing process with this degree of precision."

Comparative Analysis of Pre-trained ResNet and DenseNet Models for the Detection of Diabetic Macular Edema

A major setback in Deep Learning (DL) is that a huge amount of data is essential to render the trained model more generalizable. Constructing a higher-order model based on insufficient data has a detrimental effect on testing performance. Transfer Learning (TL) involves less training data than conventional DL approaches and offers superior decision support. Healthcare datasets of reasonable sizes are generally inappropriate for training DL models. A promising solution to the issue would be to use TL methods for the classification of medical image datasets. This paper aims at the training and evaluation of six variants of pre-trained ResNet and three variants of pre-trained DenseNet models for Diabetic Macular Edema (DME) classification employing a public retinal Optical Coherence Tomography (OCT) image dataset. Among the ResNet implementations, ResNet101V2 has delivered the highest accuracy value of 95%. And among the DenseNet implementations, DenseNet201 has yielded an exceptional classification accuracy of 99%. As all three DenseNet versions, along with the ResNet101V2 version, have produced noticeably good results (accuracy values greater than 95%), they can be used to screen the retinal OCT images of DME patients.

An Optimization Method of Deep Transfer Learning for Vegetation Segmentation under Rainy and Dry Season Differences in a Dry Thermal Valley.

Deep learning networks might require re-training for different datasets, consuming significant manual labeling and training time. Transfer learning uses little new data and training time to enable pre-trained network segmentation in relevant scenarios (e.g., different vegetation images in rainy and dry seasons); however, existing transfer learning methods lack systematicity and controllability. So, an MTPI method (Maximum Transfer Potential Index method) was proposed to find the optimal conditions in data and feature quantity for transfer learning (MTPI conditions) in this study. The four pre-trained deep networks (Seg-Net (Semantic Segmentation Networks), FCN (Fully Convolutional Networks), Mobile net v2, and Res-Net 50 (Residual Network)) using the rainy season dataset showed that Res-Net 50 had the best accuracy with 93.58% and an WIoU (weight Intersection over Union) of 88.14%, most worthy to transfer training in vegetation segmentation. By obtaining each layer's TPI performance (Transfer Potential Index) of the pre-trained Res-Net 50, the MTPI method results show that the 1000-TDS and 37-TP were estimated as the best training speed with the smallest dataset and a small error risk. The MTPI transfer learning results show 91.56% accuracy and 84.86% WIoU with 90% new dataset reduction and 90% iteration reduction, which is informative for deep networks in segmentation tasks between complex vegetation scenes.

A COMPARISON PRE-TRAINED MODELS FOR AUTOMATIC INDONESIAN LICENSE PLATE RECOGNITION

Automatic License Plate Recognition is related to the Intelligent Transportation System (ITS) that supports the road's e-law enforcement system. In the case of the Indonesian license plate, with various colour rules for font and background, and sometimes vehicle owners modify their license plate font format, this is a challenge in the image processing approach. This research utilizes pre-trained of AlexNet, VGGNet, and ResNet to determine the optimum model of Indonesian character license plate recognition. Three pre-trained approaches in CNN-based detection for reducing time for a build if model from scratch. The experiment shows that using the pre-trained ResNet model gives a better result than another two approaches. The optimum results were obtained at epoch 50 with an accuracy of 99.9% and computation time of 26 minutes. This experiment results fulfil the goal of this research.