Fine-tuning Approach Research Articles

Otolith age determination with a simple computer vision based few-shot learning method

In this study, we propose a computer vision-based few-shot learning method for otolith age determination in European plaice, Atlantic cod, Greenland halibut, and haddock. Our method outperforms prior state-of-the-art approaches, and is based on a vision encoder from CLIP as a feature extractor, which is used to train shallow models. The method is computationally efficient, as it does not require fine-tuning of deep networks, and is also data efficient, as it performs better than fine-tuning on the same data. Our results suggest that in some cases, our method can achieve the same performance as state-of-the-art finetuning approaches with up to three times less training data.

Comparison of CNN Models With Transfer Learning in the Classification of Insect Pests

Insect pests are an important problem to overcome in agriculture. The purpose of this research is to classify insect pests with the IP-102 dataset using several CNN pre-trained models and choose which model is best for classifying insect pest data. The method used is the transfer learning method with a fine-tuning approach. Transfer learning was chosen because this technique can use the features and weights that have been obtained during the previous training process. Thus, computation time can be reduced and accuracy can be increased. The models used include Xception, MobileNetV3L, MobileNetV2, DenseNet-201, and InceptionV3. Fine-tuning and freeze layer techniques are also used to improve the quality of the resulting model, making it more accurate and better suited to the problem at hand. This study uses 75,222 image data with 102 classes. The results of this study are the DenseNet-201 model with fine-tuning produces an accuracy value of 70%, MobileNetV2 66%, MobileNetV3L 68%, InceptionV3 67%, Xception 69%. The conclusion of this study is that the transfer learning method with the fine-tuning approach produces the highest accuracy value of 70% in the DenseNet-201 model.

SARS-CoV-2 Detection from Lung CT-Scan Images Using Fine Tuning Concept on Deep-CNN Pretrained Model

The problem of the spread of COVID-19 (SARS-CoV-2) is spreading fleetly and worldwide. Beforehand discovery and opinion of complaint is veritably important to ensure the right remedy so that it needs to be enforced through various practical approaches. In former studies, complaint discovery through medical imaging has started to appear and get a good delicacy of around 80 to 90 percent using machine learning. In the deep learning era, some trials get better accuracy of 95 percent using the traditional deep learning approach. Now, deep learning has developed more fleetly, especially for image classification. therefore, it's necessary to experiment with a pretrained model approach to medical images. In addition, the fine tuning approach will also be an aspect of the approach that will be carried out in this trial to be compared and to find out its effect, specifically on CT-Scan images of the lungs for the discovery of COVID 19. The results of this experiment showed that the pretrained model approach can get high accuracy. Relatively high accuracy, the smallest testing accuracy in this trial reached 94.78 percent of the Xception without fine tuning phase, this result has beaten the machine learning approach which is didn't reach 90 percent of accuracy. The best experiment testing accuracy get 97.59 percet on the VGG 16 by applying fine tuning. The results of this trial also show that the fine tuning stage (for the top 10th layers) can increase the accuracy of the model.

Low-Resource Neural Machine Translation Improvement Using Source-Side Monolingual Data

Despite the many proposals to solve the neural machine translation (NMT) problem of low-resource languages, it continues to be difficult. The issue becomes even more complicated when few resources cover only a single domain. In this paper, we discuss the applicability of a source-side monolingual dataset of low-resource languages to improve the NMT system for such languages. In our experiments, we used Wolaytta–English translation as a low-resource language. We discuss the use of self-learning and fine-tuning approaches to improve the NMT system for Wolaytta–English translation using both authentic and synthetic datasets. The self-learning approach showed +2.7 and +2.4 BLEU score improvements for Wolaytta–English and English–Wolaytta translations, respectively, over the best-performing baseline model. Further fine-tuning the best-performing self-learning model showed +1.2 and +0.6 BLEU score improvements for Wolaytta–English and English–Wolaytta translations, respectively. We reflect on our contributions and plan for the future of this difficult field of study.

A fine tuning approach using modified DenseNet model for skin cancer classification

A fine tuning approach using modified DenseNet model for skin cancer classification

A fine tuning approach using modified DenseNet model for skin cancer classification

A fine tuning approach using modified DenseNet model for skin cancer classification

Development and external validation of automated ICD-10 coding from discharge summaries using deep learning approaches

Development and external validation of automated ICD-10 coding from discharge summaries using deep learning approaches

Filtering Specialized Change in a Few-Shot Setting

The aim of change detection in remote sensing usually is not to find all differences between the observations, but rather only specific types of change, such as urban development, deforestation, or even more specialized categories like roadwork. However, often there are no large public datasets available for very fine-grained tasks, and to collect the amount of training data needed for most supervised learning methods is very costly and often prohibitive. For this reason, we formulate the problem of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">few-shot filtering</i> , where we are provided with a relatively large change detection dataset and, at test time, a few instances of one particular change type that we try to “filter out” of the learned changes. For example, we might train on data of general urban change, and, given some samples of building construction, aim to only predict instances of these on the test set, all without any explicit labels for buildings in the training data. We further investigate a fine-tuning approach to this problem and assess its performance on a public dataset that we adapt to be used in this novel setting.

Intelligent Identification of Hate Speeches to address the increased rate of Individual Mental Degeneration

Intelligent Identification of Hate Speeches to address the increased rate of Individual Mental Degeneration

Sequence-aware Knowledge Distillation for a Lightweight Event Representation

Event representation targets to model the event-reasoning process as a machine-readable format. Previous studies on event representation mostly concentrate on a sole modeling perspective and have not well investigated the scenario-level knowledge, which can cause information loss. To cope with this dilemma, we propose a unified fine-tuning architecture-based approach ( UniFA-S ) that integrates all levels of trainings, including the scenario-level knowledge. However, another challenge for existing models is the ever-increasing computation overheads, restricting the deployment ability on limited resources devices. Hence, in this article, we aim to compress the cumbersome model UniFA-S into a lighter and easy-to-deploy one without much performance damage. To this end, we propose a sequence-aware knowledge distillation model (SaKD) that employs a dynamic self-distillation on the decouple-compress-couple framework for compressing UniFA-S , which cannot only realize the model compression, but also retain the integrity of individual components. We also design two fitting strategies to address the less-supervised issue at the distillation stage. Comprehensive experiments on representation-and-inference ability-based tasks validate the effectiveness of SaKD. Compared to UniFA-S , SaKD realizes a more portable event representation model at the cost of only 1.0% performance drop in terms of accuracy or Spearman’s correlation, which is far less than other knowledge distillation models.

Weakly-supervised learning method for the recognition of potato leaf diseases.

As a crucial food crop, potatoes are highly consumed worldwide, while they are also susceptible to being infected by diverse diseases. Early detection and diagnosis can prevent the epidemic of plant diseases and raise crop yields. To this end, this study proposed a weakly-supervised learning approach for the identification of potato plant diseases. The foundation network was applied with the lightweight MobileNet V2, and to enhance the learning ability for minute lesion features, we modified the existing MobileNet-V2 architecture using the fine-tuning approach conducted by transfer learning. Then, the atrous convolution along with the SPP module was embedded into the pre-trained networks, which was followed by a hybrid attention mechanism containing channel attention and spatial attention submodules to efficiently extract high-dimensional features of plant disease images. The proposed approach outperformed other compared methods and achieved a superior performance gain. It realized an average recall rate of 91.99% for recognizing potato disease types on the publicly accessible dataset. In practical field scenarios, the proposed approach separately attained an average accuracy and specificity of 97.33% and 98.39% on the locally collected image dataset. Experimental results present a competitive performance and demonstrate the validity and feasibility of the proposed approach.

Parameter-efficient feature-based transfer for paraphrase identification

Abstract There are many types of approaches for Paraphrase Identification (PI), an NLP task of determining whether a sentence pair has equivalent semantics. Traditional approaches mainly consist of unsupervised learning and feature engineering, which are computationally inexpensive. However, their task performance is moderate nowadays. To seek a method that can preserve the low computational costs of traditional approaches but yield better task performance, we take an investigation into neural network-based transfer learning approaches. We discover that by improving the usage of parameters efficiently for feature-based transfer, our research goal can be accomplished. Regarding the improvement, we propose a pre-trained task-specific architecture. The fixed parameters of the pre-trained architecture can be shared by multiple classifiers with small additional parameters. As a result, the computational cost left involving parameter update is only generated from classifier-tuning: the features output from the architecture combined with lexical overlap features are fed into a single classifier for tuning. Furthermore, the pre-trained task-specific architecture can be applied to natural language inference and semantic textual similarity tasks as well. Such technical novelty leads to slight consumption of computational and memory resources for each task and is also conducive to power-efficient continual learning. The experimental results show that our proposed method is competitive with adapter-BERT (a parameter-efficient fine-tuning approach) over some tasks while consuming only 16% trainable parameters and saving 69-96% time for parameter update.

A fast and time-efficient glitch classification method: A deep learning-based visual feature extractor for machine learning algorithms

A fast and time-efficient glitch classification method: A deep learning-based visual feature extractor for machine learning algorithms

Prediction of oxygen requirement in patients with COVID-19 using a pre-trained chest radiograph xAI model: efficient development of auditable risk prediction models via a fine-tuning approach

Risk prediction requires comprehensive integration of clinical information and concurrent radiological findings. We present an upgraded chest radiograph (CXR) explainable artificial intelligence (xAI) model, which was trained on 241,723 well-annotated CXRs obtained prior to the onset of the COVID-19 pandemic. Mean area under the receiver operating characteristic curve (AUROC) for detection of 20 radiographic features was 0.955 (95% CI 0.938–0.955) on PA view and 0.909 (95% CI 0.890–0.925) on AP view. Coexistent and correlated radiographic findings are displayed in an interpretation table, and calibrated classifier confidence is displayed on an AI scoreboard. Retrieval of similar feature patches and comparable CXRs from a Model-Derived Atlas provides justification for model predictions. To demonstrate the feasibility of a fine-tuning approach for efficient and scalable development of xAI risk prediction models, we applied our CXR xAI model, in combination with clinical information, to predict oxygen requirement in COVID-19 patients. Prediction accuracy for high flow oxygen (HFO) and mechanical ventilation (MV) was 0.953 and 0.934 at 24 h and 0.932 and 0.836 at 72 h from the time of emergency department (ED) admission, respectively. Our CXR xAI model is auditable and captures key pathophysiological manifestations of cardiorespiratory diseases and cardiothoracic comorbidities. This model can be efficiently and broadly applied via a fine-tuning approach to provide fully automated risk and outcome predictions in various clinical scenarios in real-world practice.

Identifying Retinal Diseases on OCT Image Based on Deep Learning

Computer-aided diagnosis has the potential to replace or at least support medical personnel in their everyday responsibilities such as diagnosis, therapy, and surgery. In the area of ophthalmology, artificial intelligence approaches have been incorporated in the diagnosis of the most frequent ocular disorders, such as choroidal neovascularization (CNV), diabetic macular oedema (DMO), and DRUSEN; these illnesses pose a significant risk of vision loss. Optical coherence tomography (OCT) is an imaging technology used to diagnose the aforementioned eye disorders. It enables ophthalmologists to see the back of the eye and take various slices of the retina. The goal of this research is to automate the diagnosis of retinopathy, which includes CNV, DME, and DRUSEN. The approach employed is a deep learning-based, and transfer learning technique, applying to a public dataset of OCT pictures and two pertained neural network models VGG16 and InceptionV3, which are trained on the big database "ImageNet." That allows them to be able to extract the main features of millions of images. Furthermore, fine-tuning approaches are applied to outperform the feature extraction method, by modifying the hyperparameters. The findings showed that the VGG16 model performed better in classification than the InceptionV3 architecture, with a 0.93 accuracy.

Preterm rabbit-derived Precision Cut Lung Slices as alternative model of bronchopulmonary dysplasia in preclinical study: a morphological fine-tuning approach

Preterm rabbit-derived Precision Cut Lung Slices as alternative model of bronchopulmonary dysplasia in preclinical study: a morphological fine-tuning approach

Automated deidentification of radiology reports combining transformer and "hide in plain sight" rule-based methods.

Automated deidentification of radiology reports combining transformer and "hide in plain sight" rule-based methods.

Prompt text classifications with transformer models! An exemplary introduction to prompt-based learning with large language models

This study investigates the potential of automated classification using prompt-based learning approaches with transformer models (large language models trained in an unsupervised manner) for a domain-specific classification task. Prompt-based learning with zero or few shots has the potential to (1) make use of artificial intelligence without sophisticated programming skills and (2) make use of artificial intelligence without fine-tuning models with large amounts of labeled training data. We apply this novel method to perform an experiment using so-called zero-shot classification as a baseline model and a few-shot approach for classification. For comparison, we also fine-tune a language model on the given classification task and conducted a second independent human rating to compare it with the given human ratings from the original study. The used dataset consists of 2,088 email responses to a domain-specific problem-solving task that were manually labeled for their professional communication style. With the novel prompt-based learning approach, we achieved a Cohen’s kappa of .40, while the fine-tuning approach yields a kappa of .59, and the new human rating achieved a kappa of .58 with the original human ratings. However, the classifications from the machine learning models have the advantage that each prediction is provided with a reliability estimate allowing us to identify responses that are difficult to score. We, therefore, argue that response ratings should be based on a reciprocal workflow of machine raters and human raters, where the machine rates easy-to-classify responses and the human raters focus and agree on the responses that are difficult to classify. Further, we believe that this new, more intuitive, prompt-based learning approach will enable more people to use artificial intelligence.

A Deep Learning-Based Approach for the Diagnosis of Acute Lymphoblastic Leukemia

Leukemia is a deadly disease caused by the overproduction of immature white blood cells (WBS) in the bone marrow. If leukemia is detected at the initial stages, the chances of recovery are better. Typically, morphological analysis for the identification of acute lymphoblastic leukemia (ALL) is performed manually on blood cells by skilled medical personnel, which has several disadvantages, including a lack of medical personnel, sluggish analysis, and prediction that is dependent on the medical personnel’s expertise. Therefore, we proposed the Multi-Attention EfficientNetV2S and EfficientNetB3 state-of-the-art deep learning architectures using transfer learning-based fine-tuning approach to distinguish the normal and blast cells from microscopic blood smear images that both are pretrained on large-scale ImageNet database. We simply modified the last block of both models and added additional layers to both models. After including this Multi-Attention Mechanism, it not only reduces the model’s complexities but also generalizes its network quite well. By using the proposed technique, the accuracy has improved and the overall loss is also minimized. Our Multi-Attention EfficientNetV2S and EfficientNetB3 models achieved 99.73% and 99.25% accuracy, respectively. We have further compared the proposed model’s performance to other individual and ensemble models. Upon comparison, the proposed model outclassed the existing literature and other benchmark models, thus proving its efficiency.

Fine-tuning approach in metaheuristic algorithm to prolong wireless sensor networks nodes lifetime

Wireless sensor networks (WSN) have evolved a vibrant and lively research field. It comprises numerous wise and low-power consumption devices for gathering the contiguous atmosphere's data. However, the energy dissipation matter that distorts network lifetime remains the challenge since the sensor node battery is non-rechargeable and irreplaceable. Clustering and routing protocol has become the furthermost solutions and invariably minimizes depletion and prolongs the sensor node lifetime. Such protocols have adopted metaheuristic algorithms to secure the efficiency of the clustering and routing protocols. However, the cluster head's extensive task favors consuming and draining more energy. This study proposed a fine-tuning solution for the sensor node's population and generation sizes. It benefits from the modified problem-oriented genetic algorithm parameters in securing the sensor node lifetime. Besides, the solution works effectively to balance the load of the cluster head nodes. A set of simulations has been performed using MATLAB R2018b on the proposed solution, namely the energy efficient of genetic (EEG) algorithm and has revealed that the solution outperforms the network lifetime and cluster head load of the existing solution.