PreLora: A fine-tuning approach with low-rank matrix decomposition and prefix tuning for pre-hospital emergency text classification.

Abstract Drivable area segmentation (DAS) plays an important role in autonomous driving. Segment anything model (SAM) has recently emerged as a powerful foundation model, demonstrating remarkable potential across diverse downstream segmentation tasks through domain-specific parameter-efficient fine-tuning (PEFT). This paper explores effective adaptation strategies for applying SAM to DAS. However, existing approaches suffer from the following two limitations: 1) SAM employs a vanilla vision transformer (ViT) as its image encoder. However, the ViT struggles to extract multi-scale features without incurring substantial computational overhead; 2) current fine-tuning approaches for SAM have been found to inadequately explore traffic scene context. Thus, they are not fully optimized for DAS and leave much room for improvement. To address the above issues, we propose segment anything model for drivable area segmentation termed as DAS-SAM, a novel efficient adaption framework that fine-tunes SAM towards DAS. Our approach incorporates a lightweight, learnable network to extract multi-scale features and introduces three auxiliary learning objectives to incorporate traffic scene context. Furthermore, DAS-SAM employs mosaic image augmentation to improve robustness and generalization. Our framework is compatible with most of the existing PEFT methods, allowing for flexible integration that boosts performance. Extensive experiments on the BDD100k and Cityscapes datasets demonstrate that DAS-SAM outperforms both full fine-tuning and state-of-the-art PEFT methods.

Skin cancer, particularly melanoma, is a rapidly spreading and potentially life-threatening disease affecting humans. Melanoma typically begins on the skin's surface before penetrating deeper layers. Early detection significantly improves survival rates, with simple and cost-effective treatments yielding a 96% success rate. Traditional diagnosis methods rely on expert dermatologists, specialized equipment, and invasive biopsies. Deep learning offers advanced solutions for detecting skin cancer earlier and with high accuracy to mitigate costs and assist dermatologists. Deep Convolutional Neural Networks have shown promise in several computer vision tasks, including image classification, prompting their application in dermatology. This work focuses on leveraging three prominent DCNN architectures, DenseNet 201, VGG16, and InceptionV3, to classify skin lesions using dermoscopic images. The HAM10000 dataset was taken and divided into training and testing sets. The preprocessing methods include image normalization, scaling, and Otsu's binary thresholding segmentation and augmentation techniques were applied. We introduced two fine-tuning approaches. Firstly, the top layers of the base model are retrained. Secondly, retraining the half layers of the base models and additional layers are added to form customized CNN models. We merge these underlying models into an ensemble and hyperparameter tuning to enhance performance. The transparency and interpretability of the model are enhanced by Grad-CAM, which raises the model's dependability for clinical applications. Combining DenseNet-201, InceptionV3, and VGG16, the proposed ensemble model outperforms the individual models with a testing accuracy of 97.9%. Additionally, it exhibits a better F1-score, recall, and precision of 99.2%, demonstrating its efficacy in automated skin lesion detection.

Mathematical reasoning has long represented one of the most fundamental and challenging frontiers in artificial intelligence research. In recent years, large language models (LLMs) have achieved significant advances in this area. This survey examines the development of mathematical reasoning abilities in LLMs through two high-level cognitive phases: comprehension, where models gain mathematical understanding via diverse pretraining strategies, and answer generation, which has progressed from direct prediction to step-by-step Chain-of-Thought (CoT) reasoning. We review methods for enhancing mathematical reasoning, ranging from training-free prompting to fine-tuning approaches such as supervised fine-tuning and reinforcement learning, and discuss recent work on extended CoT and “test-time scaling”. Despite notable progress, fundamental challenges remain in terms of capacity, efficiency, and generalization. To address these issues, we highlight promising research directions, including advanced pretraining and knowledge augmentation techniques, formal reasoning frameworks, and meta-generalization through principled learning paradigms. This survey tries to provide some insights for researchers interested in enhancing reasoning capabilities of LLMs and for those seeking to apply these techniques to other domains.

The rapid expansion of deep learning in agricultural diagnostics has driven a shift toward lightweight architectures capable of operating on resource-constrained devices. While Convolutional Neural Networks (CNNs) have demonstrated success in plant disease classification, they struggle to capture global context and suffer from computational inefficiency when fine-tuning large-scale models on limited datasets. This study proposes a parameter-efficient fine-tuning (PEFT) approach using adapter modules integrated into Vision Transformer (ViT-Small-Patch16) architecture for automated tomato leaf disease classification across 10 disease classes. The method employs lightweight adapter modules comprising down-projection, ReLU activation, and up-projection layers with residual connections, inserted into each transformer block while freezing the pre-trained ViT backbone. This selective training strategy drastically reduces trainable parameters compared to full fine-tuning, mitigating overfitting risks on the PlantVillage dataset comprising 16,011 images. The model is trained using comprehensive data augmentation (HSV color adjustments, geometric transformations, and horizontal flips), stratified dataset partitioning (79.97% training, 10.03% validation, 10.00% test), and modern optimization techniques including AdamW optimizer, cosine annealing learning rate scheduler, label smoothing, gradient clipping, and early stopping. Experimental results demonstrate exceptional performance, achieving 99.81% accuracy, precision, recall, and F1-score of 0.9981, with perfect AUC-ROC and AUC-PR scores (1.0000) across all 10 disease classes. The model maintains computational efficiency with an inference time of 34.15 ms per image, making it suitable for real-time agricultural monitoring systems. Notably, minority classes such as Tomato Mosaic Virus (37 test samples) achieved perfect classification metrics, underscoring the model's robustness across imbalanced class distributions. Comparative analysis with recent tomato disease classification methods reveals that the proposed ViT-Adapter approach achieves competitive or superior performance in accuracy (99.81%), F1-score (99.81%), and inference speed (34.15 ms) while covering the broadest disease taxonomy (10 classes).

Patient-specific fine-tuning of a self-distilled transformer model for normal tissue segmentation in abdominal magnetic resonance guided adaptive radiotherapy.

Automatic question-answering systems demonstrate valuable utility in the biomedical domain, improving the precision and efficiency of clinical decision-making significantly. Despite large-scale language models achieving notable success in general domains, even outperforming human-level performance in certain aspects, they are still faced with challenges such as data privacy and scarcity in the special domain. This study explores the method for efficient fine-tuning under resource-constrained conditions in the biomedical field. We propose a multi-stage fine-tuning approach that effectively improves the performance of pre-trained language models in biomedical question-answering tasks. Specially, A multi-prompt-based contrastive learning strategy and a multi-prompt self-consistency voting module are introduced, which improve the accuracy of QA tasks. The experiments on the PubMedQA dataset under reasoning-required settings indicate that our approach outperforms domain-specific pre-training models and achieves comparable performance with GPT-4, while the number of fine-tuned parameters is much less than the total parameters of the base model.

Within the realm of software engineering, specialized tasks on code, such as program repair, present unique challenges, necessitating fine-tuning Large language models (LLMs) to unlock state-of-the-art performance. Fine-tuning approaches proposed in the literature for LLMs on program repair tasks generally overlook the need to reason about the logic behind code changes, beyond syntactic patterns in the data. High-performing fine-tuning experiments also usually come at very high computational costs. With MORepair , we propose a novel perspective on the learning focus of LLM fine-tuning for program repair: we not only adapt the LLM parameters to the syntactic nuances of the task of code transformation (objective ➊), but we also specifically fine-tune the LLM with respect to the logical reason behind the code change in the training data (objective ➋). Such a multi-objective fine-tuning will instruct LLMs to generate high-quality patches. We apply MORepair to fine-tune four open-source LLMs with different sizes and architectures. Experimental results on function-level and repository-level repair benchmarks show that the implemented fine-tuning effectively boosts LLM repair performance by 11.4% to 56.0%. We further show that our fine-tuning strategy yields superior performance compared to the state-of-the-art approaches, including standard fine-tuning, Fine-tune-CoT, and RepairLLaMA.

Dementia is a progressive neurodegenerative disease that is a major challenge to healthcare systems around the world and hence there is the need to have accurate and automated means of diagnosis. SMRI has become a useful modality for detecting neuroanatomical changes during the development of dementia, and yet, manual interpretation is tedious and prone to error. This paper will discuss a deep learning-based method of multiclass dementia classification based on a transfer learning model on the VGG-16 convolutional neural network. A big publicly available Kaggle data set comprising of about 44,000 T1-weighted brain MRI images was used, which comprised of four clinically relevant classes such as Non-Demented, Very Mild Demented, Mild Demented as well as Moderate Demented. All pictures were downsampled to the constant resolution size at 224 x 224 pixels and categorized as grayscale inputs to retain structural information and making them computationally efficient. A fine-tuning approach that was under a controlled strategy was used by unfreezing convolutional layers consecutively, allowing the detailed examination of convolutional layers parameter adaptation and generalization behaviours. The evaluation of the model performance was conducted based on accuracy metrics, learning curves, and analysis of the confusion matrix to present both quantitative and class-wise information. The final training accuracy of the proposed model was 89 percent and a validation accuracy of 76 percent which showed that the model converged well and the generalization was also good. The confusion matrix showed that Non-Demented cases were highly specific with potential difficulties likely to arise in making a distinction between the early stages of dementia since there were minor neuroanatomical overlaps.

Dental plaques are biofilms formed by microorganisms and extracellular matrix in the oral cavity. The most common oral diseases, including dental caries and periodontal diseases, are initiated by dental plaque accumulating on tooth surfaces. We develop an AI-based algorithm (AI-Plaque) to automatically segment dental plaque areas in intraoral images, enabling automatic plaque detection and quantitative severity assessment. To mitigate the limited segmentation capabilities of existing models and dataset limitations, we introduce a data preprocessing procedure, a specialized fine-tuning approach, and a novel self-training pipeline that leverages synthesized plaque images. Our approach demonstrates significant improvements in plaque segmentation and high effectiveness in plaque assessment, over several baseline methods, as evaluated by both image segmentation metrics and a custom-designed quantitative Visual Plaque Index. The AI-Plaque algorithm could empower individuals to receive timely, personalized feedback on their oral hygiene practices for dental plaque control, such as brushing and flossing, and ultimately reduce the risk of developing oral diseases.

Adenosine-to-inosine (A-to-I) RNA editing by ADAR enzymes shapes transcript fate and underpins emerging RNA editing therapeutics, yet predicting which adenosines are edited remains difficult. We introduce ADAR-GPT, a model-agnostic fine-tuning framework that adapts a GPT-class language model to classify editing at candidate sites using sequence context in standardized 201 nt windows with the target adenosine explicitly marked. We train and evaluate on GTEx liver data ([Formula: see text] samples) at a clinically relevant 15% editing threshold, using a two-stage continual fine-tuning approach where lower thresholds serve as curriculum data to progressively sharpen decision boundaries. Using sequence data, ADAR-GPT demonstrates competitive or superior performance when benchmarked against established computational approaches, including convolutional and foundation model architectures, achieving a better balance of recall, precision, and specificity alongside stronger operating-curve metrics. The approach is reproducible and portable across GPT backbones without architectural changes. Beyond accurate site classification, ADAR-GPT provides practical adenosine scoring to prioritize experimental targets and inform guide RNA design, with a framework adaptable to new datasets and model architectures.

Recent advances in spatial omics technologies have enabled transcriptome profiling at subcellular resolution. By performing cell segmentation on nuclear or membrane staining images, researchers can acquire single cell level spatial gene expression data, which in turn enables subsequent biological interpretation. Although deep learning-based segmentation models achieve high overall accuracy, their performance remains suboptimal for whole-tissue analysis, particularly in ensuring consistent segmentation accuracy across diverse cell populations. Existing fine-tuning approaches often require extensive retraining or are tailored to specific model architectures, limiting their adaptability and scalability in practical settings. To address these challenges, we present CSRefiner, a lightweight and efficient fine-tuning framework for precise whole-tissue single-cell spatial expression analysis. Our approach incorporates support for fine-tuning widely used segmentation models in the field of spatial omics, while achieving high accuracy with very limited annotated data. This study demonstrates CSRefiner’s superior performance across various staining types and its compatibility with multiple mainstream models. Combining operational simplicity with robust accuracy, our framework offers a practical solution for real-world spatial transcriptomics applications.

Direct ethanol fuel cells (DEFCs) have attracted much attention due to their high energy density and readily available fuels, but their development is limited by the poor performance of anode catalysts. Although Pd-based catalysts exhibit excellent activity in the ethanol oxidation reaction (EOR), their active site density is low, and they are easily poisoned by strong adsorption of CO* intermediates. Herein, we designed unique nanoflower-like AuPdCu nanoparticles (NPs) through a liquid-phase reflux process. Compared to conventional nanoflowers, their petals consist of numerous converging nanowires. Furthermore, the electronic structure of Pd-based catalysts can be finely tuned by alloying with Au and Cu to regulate the adsorption and desorption of CO* intermediates. As a result, the designed AuPdCu NPs demonstrated exceptional catalytic activity for the EOR, achieving a mass activity (MA) of 6.89 A mgPd-1, surpassing commercial Pd/C and some recently reported catalysts. After a 5000 s chronoamperometric stability test, they maintained a current density of 34.81 mA cm-2. Density functional theory (DFT) calculations confirm that the adsorption of the CH3CO* intermediate on AuPdCu NPs is enhanced, thereby promoting the EOR process along the C1 pathway. This ternary metal fine-tuning alloying approach presents a viable route for fabricating highly active and durable EOR materials.

The rapid growth of model scale has necessitated substantial computational resources for fine-tuning. Existing approach such as low-rank adaptation (LoRA) has sought to address the problem of handling the large updated parameters in full fine-tuning (FT). However, LoRA utilize random initialization and optimization of low-rank matrices to approximate updated weights, which can result in suboptimal convergence and an accuracy gap compared to full fine-tuning (FT). To address these issues, we propose low-rank LDU (LoLDU), a parameter-efficient fine-tuning (PEFT) approach that significantly reduces trainable parameters by 2600 times compared to regular PEFT methods while maintaining comparable performance. LoLDU leverages lower-diag-upper (LDU) decomposition to initialize low-rank matrices for faster convergence and nonsingularity. We focus on optimizing the diagonal matrix for scaling transformations. To the best of our knowledge, LoLDU has the fewest parameters among all PEFT approaches. We conducted extensive experiments across 4 instruction-following datasets, six natural language understanding (NLU) datasets, eight image classification datasets, and image generation datasets with multiple model types [LLaMA2, RoBERTa, ViT, and stable diffusion (SD)], providing a comprehensive and detailed analysis. Our open-source code can be accessed at https://anonymous.4open.science/r/LoLDU-B5A6.

Photovoltaic power forecasting based on large language models: A prompt-learning and two-stage fine-tuning approach

Machine learning models, especially neural networks, are vulnerable to adversarial attacks, where inputs are purposefully altered to induce incorrect predictions. These adversarial inputs closely resemble benign (unaltered) inputs, making them difficult to detect, and pose significant security risks in critical applications, such as autonomous vehicles, medical diagnostics, and financial transactions. Several methods exist to improve the model’s performance against these adversarial attacks, which typically modify the network architecture or training procedure. Often times, these adversarial training techniques only provide robustness against specific attack types and/or require substantial computational resources, making them impractical for real-world applications with limited resources. In this work, we propose a computationally-efficient adversarial fine-tuning approach to enhance the robustness of Convolutional Neural Networks (CNNs) against adversarial attacks and attain the same level of performance as the conventional adversarial training. More specifically, we propose to identify specific parts of the neural network model that are more vulnerable to adversarial attacks. Our analysis reveals that only a small portion of these vulnerable components accounts for a majority of the model’s errors caused by adversarial attacks. As such, we propose to selectively fine-tune these vulnerable components using different adversarial training methods to develop an effective and resource-efficient approach to improve model robustness. We empirically validate our proposed approach with varying dataset and algorithm parameters. We demonstrate that our approach can achieve similar performance as the more resource-intensive conventional adversarial training method. Received: 18 April 2025 | Revised: 4 September 2025 | Accepted: 12 November 2025 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in github.io at https://doi.org/10.1109/5.726791, reference number [3]. Author Contribution Statement Subah Karnine: Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Sadia Afrose: Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Hafiz Imtiaz: Conceptualization, Methodology, Formal analysis, Resources, Writing – original draft, Writing – review & editing, Supervision, Project administration.

Recent breakthroughs in large-scale speech models such as Whisper, Wav2Vec 2.0, and HuBERT have greatly enhanced speech processing tasks. Full fine-tuning comes at a prohibitive cost, though, which restricts their application to low- resource or real-time settings. The parameter-efficient fine-tuning (PEFT) approaches—e.g., LoRA, QLoRA, adapters, and prompt tuning—allow for compact adaptation by fine-tuning only a small subset of parameters. We review 33 studies (2021–2025) using PEFT for applications such as ASR, speaker verification, and emotion recognition. We organize methods by task, compare efficiency and accuracy, and determine prominent trends. Results indicate PEFT produces competitive results with reduced cost, enabling scalable deployment in resource-poor environments.

BackgroundAneurysmal subarachnoid hemorrhage (SAH) is a life-threatening neurological emergency with mortality rates exceeding 30%. While deep learning techniques show promise for automated SAH segmentation, their clinical application is limited by the scarcity of labeled data and challenges in cross-institutional generalization. Transfer learning from related hematoma types represents a potentially valuable but underexplored approach. Although Unet architectures remain the gold standard for medical image segmentation due to their effectiveness on limited datasets, Low-Rank Adaptation (LoRA) methods for parameter-efficient transfer learning have been rarely applied to convolutional neural networks in medical imaging contexts. The importance of SAH diagnosis and the time-intensive nature of manual annotation would benefit from automated solutions that can leverage existing multi-institutional datasets from more common conditions.MethodsWe implemented a Unet architecture pre-trained on computed tomography scans from 124 traumatic brain injury patients across multiple institutions, then fine-tuned on 30 aneurysmal SAH patients from the University of Michigan Health System using 3-fold cross-validation. We developed a novel CP-LoRA method based on tensor canonical polyadic (CP) decomposition and introduced DoRA variants (DoRA-C, convDoRA, CP-DoRA) that decompose weight matrices into magnitude and directional components. We compared these approaches against existing LoRA methods (LoRA-C, convLoRA) and standard fine-tuning strategies across different modules on a multi-view Unet model. Performance was evaluated using Dice scores stratified by hemorrhage volume, with additional assessment of predicted versus annotated blood volumes.ResultsTransfer learning from traumatic brain injury to aneurysmal SAH demonstrated feasibility with all fine-tuning approaches achieving superior performance compared to no fine-tuning (mean Dice 0.410 ± 0.26). The best-performing traditional approach was decoding module fine-tuning (Dice 0.527 ± 0.20). LoRA-based methods consistently outperformed standard Unet fine-tuning, with DoRA-C at rank 64 achieving the highest overall performance (Dice 0.572 ± 0.17). Performance varied by hemorrhage volume, with all methods showing improved accuracy for larger volumes (Dice 0.682–0.694 for volumes > 100 mL vs. Dice 0.107–0.361 for volumes < 25 mL). CP-LoRA achieved comparable performance to existing methods while using significantly fewer parameters. Over-parameterization with higher ranks (64–96) consistently yielded better performance than strictly low-rank adaptations.ConclusionsThis study demonstrates that transfer learning between hematoma types is feasible and that LoRA-based methods significantly outperform conventional Unet fine-tuning for aneurysmal SAH segmentation. The novel CP-LoRA method offers parameter efficiency advantages, while DoRA variants provide superior segmentation accuracy, particularly for small-volume hemorrhages. The finding that over-parameterization improves performance challenges traditional low-rank assumptions and suggests clinical applications may benefit from higher-rank adaptations. These results support the potential for automated SAH segmentation systems that leverage large multi-institutional traumatic brain injury datasets, potentially improving diagnostic speed and consistency when specialist expertise is unavailable.

Large language models (LLMs) are increasingly utilized for named entity recognition (NER) in health care, with significant potential to enhance symptom detection within electronic health records (EHRs). This study explores the application of LLMs to identify symptoms of anxiety and nausea/vomiting documented in the clinical notes of patients with cancer. We analyzed clinical notes from 8,490 patients diagnosed with various cancer types. Bio Clinical BERT and Bio GPT models were further pretrained on clinical text from this dataset. Two modeling strategies, fine-tuning and prompt-based learning, were implemented using Symptom-BERT and Symptom-GPT frameworks. Model performance was evaluated using F1 scores, emphasizing recognizing psychological symptoms (anxiety) and physical symptoms (nausea/vomiting). Fine-tuning with Symptom-BERT achieved the highest F1 scores, 0.989 for nausea/vomiting and 0.912 for anxiety, significantly outperforming Symptom-GPT in detection accuracy. While prompt-based learning with Symptom-GPT surpassed that of a few-shot learning, it remained less effective than fine-tuning. Fine-tuning excelled in identifying well-documented symptoms, particularly physical ones like nausea/vomiting. Using named entity recognition (NER), the study analyzed the entire dataset, detecting anxiety in 2,436 patients (28.69%) and nausea/vomiting in 3,338 patients (39.31%). While both fine-tuning and prompt-based learning approaches offer utility, fine-tuning demonstrates superior accuracy in recognizing symptoms from clinical narratives, particularly physical ones. LLM-based symptom detection can support oncology nurses and care teams by enabling earlier recognition of patient-reported symptoms documented in narrative notes. These tools offer practical value in improving symptom monitoring, care planning, and timely intervention, thereby enhancing patient-centered care in oncology settings.

Although Convolutional Neural Networks (CNNs) have delivered state-of-the-art accuracy in plant disease classification, their deployment is still hindered by data scarcity, computational cost, and architectural heterogeneity. Transfer learning from large-scale pre-trained datasets alleviates these issues, yet generic feature extraction suffers from domain shift, while indiscriminate fine-tuning risks over-fitting and elevated training budgets. To address the identified limitations, PlantClassiNet is implemented as a unified framework. This framework facilitates systematic comparative analysis of six CNN architectures, AlexNet, ResNet50, InceptionV3, MobileNetV3Small, DenseNet121 and EfficientNetB0, across three publicly available datasets: PlantVillage, PlantLeaves and Eggplant. Two alternative fine-tuning approaches are proposed: ‌Adaptive Fine-tuning (AdapFitu)‌, which adaptively determines the depth of unfrozen layers, learning rates, and reinitializes selected layers, and a ‌fixed-parameter baseline‌, which trains only the newly added classifier while keeping the convolutional backbone frozen and unfreezes a fixed number of network layers for retraining. Extensive experiments demonstrate that large models AlexNet, ResNet50, and Inceptionv3 achieve test accuracy exceeding 98.74% on the sizable PlantVillage dataset, whereas lightweight counterparts MobileNetV3Small, DenseNet121, and EfficientNetB0 achieve high accuracy of 99.79% ± 0.21% on the smaller Eggplant collection after fine-tuning.