Fine-tuning Technique Research Articles

Real-GPT: Efficiently Tailoring LLMs for Informed Decision-Making in the Real Estate Industry

Lately, large language models (LLMs) such as ChatGPT and Llama have gained significant attention. These models demonstrate a remarkable capability to solve complex tasks, drawing knowledge primarily from a generalized database rather than niche subject areas. Consequently, there has been a growing demand for domain-specific LLMs tailored to social and natural sciences, such as BloombergGPT or BioGPT. In this study, we present a domain-specific LLM focused on real estate: Real-GPT. The model is based on the parameter-efficient fine-tuning technique known as quantized low-rank adaptation (QLoRA) applied to Mistral 7B. To create a comprehensive fine-tuning dataset, we compiled a curated 21,000 self-instruction dataset sourced from 670 scientific papers, market research, scholarly articles, and real estate books. To assess the efficacy of Real-GPT, we devised a set of 1046 multiple-choice questions to gauge the real estate knowledge of the models. Furthermore, we tested Real-GPT in a real investment scenario. Despite its compact size, our model significantly outperforms GPT 3.5 and Mistral 7B. Hence, the model not only showcases superior performance but also illustrates its capacity to facilitate investment decisions or interpret current market data. This development showcases the potential of LLMs to revolutionize real estate analysis and decision-making.

PEL-PVP: Application of plant vacuolar protein discriminator based on PEFT ESM-2 and bilayer LSTM in an unbalanced dataset

Plant vacuoles, play a crucial role in maintaining cellular stability, adapting to environmental changes, and responding to external pressures. The accurate identification of vacuolar proteins (PVPs) is crucial for understanding the biosynthetic mechanisms of intracellular vacuoles and the adaptive mechanisms of plants. In order to more accurately identify vacuole proteins, this study developed a new predictive model PEL-PVP based on ESM-2. Through this study, the feasibility and effectiveness of using advanced pre-training models and fine-tuning techniques for bioinformatics tasks were demonstrated, providing new methods and ideas for plant vacuolar protein research. In addition, previous datasets for vacuolar proteins were balanced, but imbalance is more closely related to the actual situation. Therefore, this study constructed an imbalanced dataset UB-PVP from the UniProt database，helping the model better adapt to the complexity and uncertainty in real environments, thereby improving the model's generalization ability and practicality. The experimental results show that compared with existing recognition techniques, achieving significant improvements in multiple indicators, with 6.08 %, 13.51 %, 11.9 %, and 5 % improvements in ACC, SP, MCC, and AUC, respectively. The accuracy reaches 94.59 %, significantly higher than the previous best model GraphIdn. This provides an efficient and precise tool for the study of plant vacuole proteins.

DiffPLF: A conditional diffusion model for probabilistic forecasting of EV charging load

Due to the vast electric vehicle (EV) penetration to distribution grid, charging load forecasting is essential to promote charging station operation and demand-side management. However, the stochastic charging behaviors and associated exogenous factors render future charging load patterns quite volatile and hard to predict. Accordingly, we devise a novel Diffusion model termed DiffPLF for Probabilistic Load Forecasting of EV charging, which can explicitly approximate the predictive load distribution conditioned on historical data and related covariates. Specifically, we leverage a denoising diffusion model, which can progressively convert the Gaussian prior to real time-series data by learning a reversal of the diffusion process. Besides, we couple such diffusion model with a cross-attention-based conditioning mechanism to execute conditional generation for possible charging demand profiles. We also propose a task-informed fine-tuning technique to better adapt DiffPLF to the probabilistic time-series forecasting task and acquire more accurate and reliable predicted intervals. Finally, we conduct multiple experiments to validate the superiority of DiffPLF to predict complex temporal patterns of erratic charging load and carry out controllable generation based on certain covariate. Results demonstrate that we can attain a notable rise of 39.58% and 49.87% on MAE and CRPS respectively compared to the conventional method.

Towards Reliable Healthcare LLM Agents: A Case Study for Pilgrims during Hajj

There is a pressing need for healthcare conversational agents with domain-specific expertise to ensure the provision of accurate and reliable information tailored to specific medical contexts. Moreover, there is a notable gap in research ensuring the credibility and trustworthiness of the information provided by these healthcare agents, particularly in critical scenarios such as medical emergencies. Pilgrims come from diverse cultural and linguistic backgrounds, often facing difficulties in accessing medical advice and information. Establishing an AI-powered multilingual chatbot can bridge this gap by providing readily available medical guidance and support, contributing to the well-being and safety of pilgrims. In this paper, we present a comprehensive methodology aimed at enhancing the reliability and efficacy of healthcare conversational agents, with a specific focus on addressing the needs of Hajj pilgrims. Our approach leverages domain-specific fine-tuning techniques on a large language model, alongside synthetic data augmentation strategies, to optimize performance in delivering contextually relevant healthcare information by introducing the HajjHealthQA dataset. Additionally, we employ a retrieval-augmented generation (RAG) module as a crucial component to validate uncertain generated responses, which improves model performance by 5%. Moreover, we train a secondary AI agent on a well-known health fact-checking dataset and use it to validate medical information in the generated responses. Our approach significantly elevates the chatbot’s accuracy, demonstrating its adaptability to a wide range of pilgrim queries. We evaluate the chatbot’s performance using quantitative and qualitative metrics, highlighting its proficiency in generating accurate responses and achieve competitive results compared to state-of-the-art models, in addition to mitigating the risk of misinformation and providing users with trustworthy health information.

Explainable artificial intelligence in deep learning-based detection of aortic elongation on chest X-ray images.

Aortic elongation can result from age-related changes, congenital factors, aneurysms, or conditions affecting blood vessel elasticity. It is associated with cardiovascular diseases and severe complications like aortic aneurysms and dissection. We assess qualitatively and quantitatively explainable methods to understand the decisions of a deep learning model for detecting aortic elongation using chest X-ray (CXR) images. In this study, we evaluated the performance of deep learning models (DenseNet and EfficientNet) for detecting aortic elongation using transfer learning and fine-tuning techniques with CXR images as input. EfficientNet achieved higher accuracy (86.7% 2.1), precision (82.7% 2.7), specificity (89.4% 1.7), F1 score (82.5% 2.9), and area under the receiver operating characteristic (92.7% 0.6) but lower sensitivity (82.3% 3.2) compared with DenseNet. To gain insights into the decision-making process of these models, we employed gradient-weighted class activation mapping and local interpretable model-agnostic explanations explainability methods, which enabled us to identify the expected location of aortic elongation in CXR images. Additionally, we used the pixel-flipping method to quantitatively assess the model interpretations, providing valuable insights into model behaviour. Our study presents a comprehensive strategy for analysing CXR images by integrating aortic elongation detection models with explainable artificial intelligence techniques. By enhancing the interpretability and understanding of the models' decisions, this approach holds promise for aiding clinicians in timely and accurate diagnosis, potentially improving patient outcomes in clinical practice.

Transfer learning for sparse variable selection in high-dimensional regression from quadratic measurement

In signal processing, the functional relationship between the unknown parameters and response variables extends to the nonlinear forms. A representative model is the quadratic measurement regression. In high-dimensional scenarios, model complexity poses challenges for estimation and variable selection. In this study, we propose a generic transfer learning algorithm and an automatic transferable source detection algorithm to identify helpful sources for the quadratic measurement regression model. The core idea of the general algorithm is based on pre-training and fine-tuning techniques. To our knowledge, this is the first study that introduces transfer learning into this model. Numerous simulation experiments demonstrated the effectiveness of the proposed transfer learning algorithm. In simulation experiments, the proposed transfer algorithm reduced the square-sum error of the results to approximately 1. On a real-world speech dataset, the proposed algorithm significantly improved the accuracy of the reconstructed signals. In the best-case scenario, the minimum mean square error was reduced to 1/4th of that without transfer. The code is available at https://github.com/Sarkovqx/Transfer-QMR.

A distinguished deep learning method for gear fault classification using time–frequency representation

Fault diagnosis of gearboxes has attracted increasing interest in recent decades due to their ubiquity and importance in the industry. Modern research trends focus on developing a diagnosis system that works automatically with the application of artificial intelligence. These previous studies have used the Deep Learning (DL) network without adequately addressing noise of the input data, requiring more data to achieve effective training. Thus, this work proposes a novel Transfer Learning method using the time–frequency representation of gear vibration signals, which enables more accurate classification in complex working conditions and reduces necessary input data to train. Using fine-tuning techniques proposed in this paper requires only a limited data set while ensuring acceptable classification results. An experiment test rig within different gear faults and load conditions was set up to evaluate the algorithm’s effectiveness.

WSPolyp-SAM: Weakly Supervised and Self-Guided Fine-Tuning of SAM for Colonoscopy Polyp Segmentation

Ensuring precise segmentation of colorectal polyps holds critical importance in the early diagnosis and treatment of colorectal cancer. Nevertheless, existing deep learning-based segmentation methods are fully supervised, requiring extensive, precise, manual pixel-level annotation data, which leads to high annotation costs. Additionally, it remains challenging to train large-scale segmentation models when confronted with limited colonoscopy data. To address these issues, we introduce the general segmentation foundation model—the Segment Anything Model (SAM)—into the field of medical image segmentation. Fine-tuning the foundation model is an effective approach to tackle sample scarcity. However, current SAM fine-tuning techniques still rely on precise annotations. To overcome this limitation, we propose WSPolyp-SAM, a novel weakly supervised approach for colonoscopy polyp segmentation. WSPolyp-SAM utilizes weak annotations to guide SAM in generating segmentation masks, which are then treated as pseudo-labels to guide the fine-tuning of SAM, thereby reducing the dependence on precise annotation data. To improve the reliability and accuracy of pseudo-labels, we have designed a series of enhancement strategies to improve the quality of pseudo-labels and mitigate the negative impact of low-quality pseudo-labels. Experimental results on five medical image datasets demonstrate that WSPolyp-SAM outperforms current fully supervised mainstream polyp segmentation networks on the Kvasir-SEG, ColonDB, CVC-300, and ETIS datasets. Furthermore, by using different amounts of training data in weakly supervised and fully supervised experiments, it is found that weakly supervised fine-tuning can save 70% to 73% of annotation time costs compared to fully supervised fine-tuning. This study provides a new perspective on the combination of weakly supervised learning and SAM models, significantly reducing annotation time and offering insights for further development in the field of colonoscopy polyp segmentation.

Real-time quantitative detection of hydrocolloid adulteration in meat based on Swin Transformer and smartphone.

Hydrocolloids are widely used in meat products as common food additives. However, research has indicated that excessive consumption of these hydrocolloids may have potential health implications. Currently, consumers mainly rely on sensory evaluation to identify hydrocolloid adulteration in meat products. Although many studies on quantitative detection of hydrocolloids have been conducted by biochemical methods in laboratory environments, there is currently a lack of effective tools for consumers and regulators to obtain real-time and reliable information on hydrocolloid adulteration. To address this challenge, a smartphone-based computer vision method was developed to quantitatively detect carrageenan adulteration in beef in this work. Specifically, Swin Transformer models, along with pre-training and fine-tuning techniques, were used to successfully automate the classification of beef into nine different levels of carrageenan adulteration, ranging from 0% to 20%. Among the tested models, Swin-Tiny (Swin-T) achieved the highest trade-off performance, with a Top-1 accuracy of 0.997, a detection speed of 3.2ms, and a model size of 103.45 Mb. Compared to computer vision, the electrochemical impedance spectroscopy achieved a lower accuracy of 0.792 and required a constant temperature environment and a waiting time of around 30 min for data stabilization. In addition, Swin-T model was also capable of distinguishing between different types of hydrocolloids with a Top-1 accuracy of 0.975. This study provides consumers and regulators with a valuable tool to obtain real-time quantitative information about meat adulteration anytime, anywhere. PRACTICAL APPLICATION: This research provides a practical solution for regulators and consumers to non-destructively and quantitatively detect the content and type of hydrocolloids in beef in real-time using smartphones. This innovation has the potential to significantly reduce the costs associated with meat quality testing, such as the use of chemical reagents and expensive instruments.

Optimization techniques for sentiment analysis based on LLM (GPT-3)

With the rapid development of natural language processing (NLP) technology, large-scale pre-trained language models such as GPT-3 have become a popular research object in NLP field. This paper aims to explore sentiment analysis optimization techniques based on large pre-trained language models such as GPT-3 to improve model performance and effect and further promote the development of natural language processing (NLP). By introducing the importance of sentiment analysis and the limitations of traditional methods, GPT-3 and Fine-tuning techniques are introduced in this paper, and their applications in sentiment analysis are explained in detail. The experimental results show that the Fine-tuning technique can optimize GPT-3 model and obtain good performance in sentiment analysis task. This study provides an important reference for future sentiment analysis using large-scale language models.

Benchmarking Llama 3 70B for Code Generation: A Comprehensive Evaluation

This study benchmarks the capabilities of Llama 3 70B, a 70-billion parameter large language model (LLM), for code generation tasks. To effectively train and fine-tune this massive model, we integrate PyTorch Fully Sharded Data Parallel (FSDP) [1], [2] for distributed training and Quantized Low-Rank Adaptation (Q-LoRA) [7] for efficient fine-tuning. We address challenges associated with distributed training, including communication overhead and synchronization complexities, through optimization strategies like gradient accumulation, optimizer state sharding, and mixed precision training. Additionally, we employ advanced training techniques such as Curriculum Learning, Dynamic Batch Sizing, and Adaptive Optimization Algorithms to enhance model performance and training efficiency. Our primary focus is evaluating the performance of the fine-tuned Llama 3 70B model on two widely-recognized code generation benchmarks: HumanEval [8] and MBPP [9]. HumanEval assesses the model's ability to translate natural language problem descriptions into functionally correct code, while MBPP evaluates its proficiency in solving complex programming problems by generating accurate Python code. We present detailed performance results on these benchmarks, analyzing the model's strengths and limitations in various code generation scenarios. Furthermore, we compare the impact of our training and fine-tuning methodologies on scalability, memory efficiency, and training speed, demonstrating the feasibility and efficiency of our approach. This benchmark study offers valuable insights for researchers and practitioners exploring the application of LLMs for code generation. It provides a comprehensive evaluation of Llama 3 70B's capabilities, sheds light on the effectiveness of various training and fine-tuning techniques, and emphasizes the importance of rigorous benchmark evaluation in driving progress within this rapidly evolving field.

Deep learning based fault detection of automobile dry clutch system using spectrogram plots

Dry friction clutches are extremely important in the context of power transmission systems. Continuous exposure to extreme heat and loading makes clutch extremely susceptible to various faults. The timely detection and diagnosis of such faults are of utmost importance to prevent any damage to internal components and also helps in avoiding transmission system failures. In this research study, a novel approach that leverages the power of transfer learning (a famous deep learning technique) is proposed to diagnose multiple types of clutch faults including, worn release fingers, fractured pressure plates, deteriorated pressure plates, loss of friction material and distorted tangential strips using spectrogram plots. To train and validate the diagnostic system, vibration readings were taken from a specially designed test rig with the help of piezoelectric accelerometer while the clutch system was operated under different load conditions of 0 (no load), 5 and 10 kg This procedure of data collection was then repeated to acquire the vibration data for all of the fault conditions by replacing the good with fault components individually. These vibration signals were further processed and transformed into spectrogram plots that serves as the input data for the deep learning models considered. Fine-tuning techniques were applied on pretrained networks to maximise the prediction accuracy of the models to effectively determine and diagnose faults in the clutch system. For this study 12 pre-trained networks were chosen namely, Xception, InceptionResNet, DenseNet, AlexNet, VGG16, GoogLeNet, VGG19, ResNet101, ResNet50, InceptionV3, MobileNetV2 and ShuffleNet. To optimize the performance of deep learning models, a systematic adjustment of hyperparameters such as the train-test split ratio, learning rate, optimizer and batch size for each network model was carried out. Through careful experimentation and analysis, significant improvements in fault classification accuracy were achieved thereby enhancing the reliability and effectiveness of the diagnostic system. From the results it was noted that 100% classification accuracy was displayed by AlexNet (for the no load condition and the 10 kg load condition) and GoogLeNet (for 5 kg load condition) with extremely low computation times.

Bridging pre-trained models to continual learning: A hypernetwork based framework with parameter-efficient fine-tuning techniques

Modern techniques of pre-training and fine-tuning have significantly improved the performance of models on downstream tasks. However, this improvement faces challenges when pre-trained models encounter the necessity to adapt sequentially to multiple downstream tasks within the context of continuously shifting training data. In this study, we aim to leverage the general capabilities of pre-trained models for knowledge sharing across different tasks while endow them with the capability for continuous learning. To this end, we propose a Hypernetwork-based Parameter Efficient Fine-Tuning (HyperPEFT) framework. Utilizing a pre-trained Vision Transformer (ViT) as the backbone, HyperPEFT is capable of incorporating various PEFT techniques, enabling the pre-trained ViT to adapt to diverse downstream tasks. The core of our method lies in the application of hypernetworks, which efficiently encapsulate task-specific information, significantly reducing task interference and fortifying the model against catastrophic forgetting. The adoption PEFT techniques allows for precise adjustments to the pre-trained models, enhancing their performance for each specific task. Moreover, this strategy employs a shared hypernetwork to make task-specific adjustments, thereby facilitating knowledge sharing across different tasks for pre-trained models. The extensive experiments reveal that our method effectively mitigates catastrophic forgetting, outperforms comparison methods, and uncovers latent associations among tasks. Overall, this study introduces a unified strategy that synergistically blends the general capabilities of pre-trained models with the necessary adaptability for continual learning scenarios.

Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification.

Adapters and Low-Rank Adaptation (LoRA) are parameter-efficient fine-tuning techniques designed to make the training of language models more efficient. Previous results demonstrated that these methods can even improve performance on some classification tasks. This paper complements existing research by investigating how these techniques influence classification performance and computation costs compared to full fine-tuning. We focus specifically on multilingual text classification tasks (genre, framing, and persuasion techniques detection; with different input lengths, number of predicted classes and classification difficulty), some of which have limited training data. In addition, we conduct in-depth analyses of their efficacy across different training scenarios (training on the original multilingual data; on the translations into English; and on a subset of English-only data) and different languages. Our findings provide valuable insights into the applicability of parameter-efficient fine-tuning techniques, particularly for multilabel classification and non-parallel multilingual tasks which are aimed at analysing input texts of varying length.

Strawberry disease detection using transfer learning of deep convolutional neural networks

The impact of disease on strawberry quality and yield holds considerable significance, prompting researchers to explore effective methodologies for disease detection in strawberries. Among these, deep learning has emerged as a pivotal approach. In this regard, this research explored the utilization of transfer learning in deep convolutional neural networks (CNNs) to identify various strawberry diseases. Specifically, we utilized models pre-trained on the ImageNet dataset, namely VGG19, Inception V3, ResNet50, and DenseNet121 architectures, employing both fine-tuning and feature extraction techniques of transfer learning and consequently compared to the models without transfer learning. The target diseases for identification included angular leaf spot, anthracnose, gray mold, and powdery mildew on both fruit and leaves. The study outcomes revealed that Resnet-50 consistently achieved the highest accuracy across all three configurations, achieving its peak accuracy at 94.4 %, followed by Densenet-121 with an accuracy of 94.1 % attained through fine-tuning. These results highlighted the superior performance of fine-tuned models over using these models solely as feature extractors for identifying strawberry diseases. Furthermore, this study revealed that the application of transfer learning substantially reduced training time and resulted in a lower count of trainable parameters than models trained without transfer learning. These outcomes strongly endorse the practicality and effectiveness of employing transfer learning techniques for precise strawberry disease identification. Additionally, further research can explore the application of transfer learning to a broader range of crops and diseases, potentially enhancing agricultural disease detection methodologies.

GPT for medical entity recognition in Spanish

AbstractIn recent years, there has been a remarkable surge in the development of Natural Language Processing (NLP) models, particularly in the realm of Named Entity Recognition (NER). Models such as BERT have demonstrated exceptional performance, leveraging annotated corpora for accurate entity identification. However, the question arises: Can newer Large Language Models (LLMs) like GPT be utilized without the need for extensive annotation, thereby enabling direct entity extraction? In this study, we explore this issue, comparing the efficacy of fine-tuning techniques with prompting methods to elucidate the potential of GPT in the identification of medical entities within Spanish electronic health records (EHR). This study utilized a dataset of Spanish EHRs related to breast cancer and implemented both a traditional NER method using BERT, and a contemporary approach that combines few shot learning and integration of external knowledge, driven by LLMs using GPT, to structure the data. The analysis involved a comprehensive pipeline that included these methods. Key performance metrics, such as precision, recall, and F-score, were used to evaluate the effectiveness of each method. This comparative approach aimed to highlight the strengths and limitations of each method in the context of structuring Spanish EHRs efficiently and accurately.The comparative analysis undertaken in this article demonstrates that both the traditional BERT-based NER method and the few-shot LLM-driven approach, augmented with external knowledge, provide comparable levels of precision in metrics such as precision, recall, and F score when applied to Spanish EHR. Contrary to expectations, the LLM-driven approach, which necessitates minimal data annotation, performs on par with BERT’s capability to discern complex medical terminologies and contextual nuances within the EHRs. The results of this study highlight a notable advance in the field of NER for Spanish EHRs, with the few shot approach driven by LLM, enhanced by external knowledge, slightly edging out the traditional BERT-based method in overall effectiveness. GPT’s superiority in F-score and its minimal reliance on extensive data annotation underscore its potential in medical data processing.

Classification of Diabetic Retinopathy using Pre-Trained Deep Learning Models

Diabetic Retinopathy (DR) stands as the leading cause of blindness globally, particularly affecting individuals between the ages of 20 and 70. This paper presents a Computer- Aided Diagnosis (CAD) system designed for the automatic classification of retinal images into five distinct classes: Normal, Mild, Moderate, Severe, and Proliferative Diabetic Retinopathy (PDR). The proposed system leverages Convolutional Neural Networks (CNNs) employing pre-trained deep learning models. Through the application of fine-tuning techniques, our model is trained on fundus images of diabetic retinopathy with resolutions of 350x350x3 and 224x224x3. Experimental results obtained on the Kaggle platform, utilizing resources comprising 4 CPUs, 17 GB RAM, and 1 GB Disk, demonstrate the efficacy of our approach. The achieved Area Under the Curve (AUC) values for CNN, MobileNet, VGG-16, InceptionV3, and InceptionResNetV2 models are 0.50, 0.70, 0.53, 0.63, and 0.69, respectively.

Meta-Learning Enhanced Trade Forecasting: A Neural Framework Leveraging Efficient Multicommodity STL Decomposition

In the dynamic global trade environment, accurately predicting trade values of diverse commodities is challenged by unpredictable economic and political changes. This study introduces the Meta-TFSTL framework, an innovative neural model that integrates Meta-Learning Enhanced Trade Forecasting with efficient multicommodity STL decomposition to adeptly navigate the complexities of forecasting. Our approach begins with STL decomposition to partition trade value sequences into seasonal, trend, and residual elements, identifying a potential 10-month economic cycle through the Ljung–Box test. The model employs a dual-channel spatiotemporal encoder for processing these components, ensuring a comprehensive grasp of temporal correlations. By constructing spatial and temporal graphs leveraging correlation matrices and graph embeddings and introducing fused attention and multitasking strategies at the decoding phase, Meta-TFSTL surpasses benchmark models in performance. Additionally, integrating meta-learning and fine-tuning techniques enhances shared knowledge across import and export trade predictions. Ultimately, our research significantly advances the precision and efficiency of trade forecasting in a volatile global economic scenario.

Research on the queuing theory in practical applications

In the face of mounting global urbanization and digitization trends, the need for advanced tools for city planners and system managers becomes increasingly paramount to ensure seamless infrastructure operations. Among the arsenal of available tools, queuing theory emerges as a standout, offering invaluable predictions and strategies for a broad spectrum of situations. This article delves into the nuanced applications of queuing theory, with a specific lens on network communications and urban space planning. Drawing from a rich tapestry of academic sources, the narrative weaves together core principles to shape models that mirror real-world situations. At the heart of this exploration lies a deep dive into solutions that tackle network delay challenges, fine-tuning techniques for 6TiSCH resource allocation, and the subtle art of queue design at railway ticket counters. These instances highlight the adaptability and immediate relevance of queuing theory across various sectors. Those in the fields of design, system architecture, and urban planning will find this read enlightening. By leveraging the insights offered, decision-makers can pave the way for optimized system functionalities and heightened user experiences, vital in an era dominated by urban sprawl and digital transformation.

Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models

The popularity of pre-trained large models has revolutionized downstream tasks across diverse fields, such as language, vision, and multi-modality. To minimize the adaption cost for downstream tasks, many Parameter-Efficient Fine-Tuning (PEFT) techniques are proposed for language and 2D image pre-trained models. However, the specialized PEFT method for 3D pre-trained models is still under-explored. To this end, we introduce Point-PEFT, a novel framework for adapting point cloud pre-trained models with minimal learnable parameters. Specifically, for a pre-trained 3D model, we freeze most of its parameters, and only tune the newly added PEFT modules on downstream tasks, which consist of a Point-prior Prompt and a Geometry-aware Adapter. The Point-prior Prompt adopts a set of learnable prompt tokens, for which we propose to construct a memory bank with domain-specific knowledge, and utilize a parameter-free attention to enhance the prompt tokens. The Geometry-aware Adapter aims to aggregate point cloud features within spatial neighborhoods to capture fine-grained geometric information through local interactions. Extensive experiments indicate that our Point-PEFT can achieve better performance than the full fine-tuning on various downstream tasks, while using only 5% of the trainable parameters, demonstrating the efficiency and effectiveness of our approach. Code is released at https://github.com/Ivan-Tang-3D/Point-PEFT.