Fine-tuned Model Research Articles

Adaptafood: an intelligent system to adapt recipes to specialised diets and healthy lifestyles

This paper presents AdaptaFood, a system to adapt recipes to specific dietary constraints. This is a common societal issue due to various dietary needs arising from medical conditions, allergies, or nutritional preferences. AdaptaFood provides recipe adaptations from two inputs: a recipe image (a fine-tuned image-captioning model allows us to extract the ingredients) or a recipe object (we extract the ingredients from the recipe features). For the adaptation, we propose to use an attention-based language sentence model based on BERT to learn the semantics of the ingredients and, therefore, discover the hidden relations among them. Specifically, we use them to perform two tasks: (1) align the food items from several sources to expand recipe information; (2) use the semantic features embedded in the representation vector to detect potential food substitutes for the ingredients. The results show that the model successfully learns domain-specific knowledge after re-training it to the food computing domain. Combining this acquired knowledge with the adopted strategy for sentence representation and food replacement enables the generation of high-quality recipe versions and dealing with the heterogeneity of different-origin food data.

A pediatric emergency prediction model using natural language process in the pediatric emergency department

This study developed a predictive model using deep learning (DL) and natural language processing (NLP) to identify emergency cases in pediatric emergency departments. It analyzed 87,759 pediatric cases from a South Korean tertiary hospital (2012–2021) using electronic medical records. Various NLP models, including four machine learning (ML) models with Term Frequency-Inverse Document Frequency (TF-IDF) and two DL models based on the KM-BERT framework, were trained to differentiate emergency cases using clinician transcripts. Gradient Boosting, among the ML models, performed best with an AUROC of 0.715, AUPRC of 0.778, and F1-score of 0.677. DL models, especially the fine-tuned KM-BERT model, showed superior performance, achieving an AUROC of 0.839, AUPRC of 0.879, and F1-score of 0.773. Shapley-based explanations provided insights into model predictions, underlining the potential of these technologies in medical decision-making. This study demonstrates the potential of advanced DL techniques for NLP in emergency medical settings, offering a more precise and efficient approach to managing healthcare resources and improving patient outcomes.

A preoperative predictive model based on multi-modal features to predict pathological complete response after neoadjuvant chemoimmunotherapy in esophageal cancer patients

BackgroundThis study aimed to develop a multi-modality model by incorporating pretreatment computed tomography (CT) radiomics and pathomics features along with clinical variables to predict pathologic complete response (pCR) to neoadjuvant chemoimmunotherapy in patients with locally advanced esophageal cancer (EC).MethodA total of 223 EC patients who underwent neoadjuvant chemoimmunotherapy followed by surgical intervention between August 2021 and December 2023 were included in this study. Radiomics features were extracted from contrast-enhanced CT images using PyrRadiomics, while pathomics features were derived from whole-slide images (WSIs) of pathological specimens using a fine-tuned deep learning model (ResNet-50). After feature selection, three single-modality prediction models and a combined multi-modality model integrating two radiomics features, 11 pathomics features, and two clinicopathological features were constructed using the support vector machine (SVM) algorithm. The performance of the models were evaluated using receiver operating characteristic (ROC) analysis, calibration plots, and decision curve analysis (DCA). Shapley values were also utilized to explain the prediction model.ResultsThe predictive capability of the multi-modality model in predicting pCR yielded an area under the curve (AUC) of 0.89 (95% confidence interval [CI], 0.75-1.00), outperforming the radiomics model (AUC 0.70 [95% CI 0.54-0.85]), pathomics model (AUC 0.77 [95% CI 0.53-1.00]), and clinical model (AUC 0.63 [95% CI 0.46-0.80]). Additionally, both the calibration plot and DCA curves support the clinical utility of the integrated multi-modality model.ConclusionsThe combined multi-modality model we propose can better predict the pCR status of esophageal cancer and help inform clinical treatment decisions.

Generating conceptual landscape design via text-to-image generative AI model

This study explores the integration of text-to-image generative AI, particularly Stable Diffusion, in conjunction with ControlNet and LoRA models in conceptual landscape design. Traditional methods in landscape design are often time-consuming and limited by the designer’s individual creativity, also often lacking efficiency in the exploration of diverse design solutions. By leveraging AI tools, we demonstrate a workflow that efficiently generates detailed and visually coherent landscape designs, including natural parks, city plazas, and courtyard gardens. Through both qualitative and quantitative evaluations, our results indicate that fine-tuned models produce superior designs compared to non-fine-tuned models, maintaining spatial consistency, control over scale, and relevant landscape elements. This research advances the efficiency of conceptual design processes and underscores the potential of AI in enhancing creativity and innovation in landscape architecture.

Enhancing Colony Detection of Microorganisms in Agar Dishes Using SAM-Based Synthetic Data Augmentation in Low-Data Scenarios

In many medical and pharmaceutical processes, continuous hygiene monitoring relies on manual detection of microorganisms in agar dishes by skilled personnel. While deep learning offers the potential for automating this task, it often faces limitations due to insufficient training data, a common issue in colony detection. To address this, we propose a simple yet efficient SAM-based pipeline for Copy-Paste data augmentation to enhance detection performance, even with limited data. This paper explores a method where annotated microbial colonies from real images were copied and pasted into empty agar dish images to create new synthetic samples. These new samples inherited the annotations of the colonies inserted into them so that no further labeling was required. The resulting synthetic datasets were used to train a YOLOv8 detection model, which was then fine-tuned on just 10 to 1000 real images. The best fine-tuned model, trained on only 1000 real images, achieved an mAP of 60.6, while a base model trained on 5241 real images achieved 64.9. Although far fewer real images were used, the fine-tuned model performed comparably well, demonstrating the effectiveness of the SAM-based Copy-Paste augmentation. This approach matches or even exceeds the performance of the current state of the art in synthetic data generation in colony detection and can be expanded to include more microbial species and agar dishes.

Urinary Bladder Acute Inflammations and Nephritis of the Renal Pelvis: Diagnosis Using Fine-Tuned Large Language Models

Background: Large language models (LLMs) have seen a significant boost recently in the field of natural language processing (NLP) due to their capabilities in analyzing words. These autoregressive models prove robust in classification tasks where texts need to be analyzed and classified. Objectives: In this paper, we explore the power of base LLMs such as Generative Pre-trained Transformer 2 (GPT-2), Bidirectional Encoder Representations from Transformers (BERT), Distill-BERT, and TinyBERT in diagnosing acute inflammations of the urinary bladder and nephritis of the renal pelvis. Materials and Methods: the LLMs were trained and tested using supervised fine-tuning (SFT) on a dataset of 120 examples that include symptoms that may indicate the occurrence of these two conditions. Results: By employing a supervised fine-tuning method and carefully crafted prompts to present the data, we demonstrate the feasibility of using minimal training data to achieve a reasonable diagnostic, with overall testing accuracies of 100%, 100%, 94%, and 79%, for GPT-2, BERT, Distill-BERT, and TinyBERT, respectively.

Review of Gen AI Models for Financial Risk Management

In this paper, we propose and demonstrate a prototype for leveraging Generative AI (GenAI) in financial risk analysis, specifically focusing on fine-tuning GPT models with proprietary data. Financial risk modeling, development, validation, and approval require not only advanced AI techniques but also careful implementation, given the vast and complex datasets involved in such tasks. The research underscores the critical importance of human oversight in mitigating potential failures that can arise from fully automated mathematical models. The study explores the application of Large Language Models (LLMs) in various financial risk domains, such as credit risk assessment, market risk forecasting, and anomaly detection. While synthetic data generation is excluded from this work, the research highlights the use of zero-shot classification leveraging Hugging Face models and OpenAI tools. ChatGPT achieved over 70% accuracy in generating relevant questions and demonstrated 60% correctness in classification tasks. Additionally, we present a prototype pipeline that integrates GenAI capabilities into financial workflows, which is implementable on small-scale computing systems. This includes backend testing via Flask and rapid prototyping using cURL commands, offering a practical approach to testing and deploying models. By fine-tuning GenAI with domain-specific data and optimizing decision-making processes, this research highlights the transformative potential of integrating generative AI into financial risk management. The study provides insights into enhancing model efficiency, regulatory compliance, and scalability. Moreover, it addresses critical challenges such as handling large datasets and ensuring ethical AI use in decision-making systems. This work contributes to advancing the adoption of GenAI in financial analytics, paving the way for innovative, robust, and efficient methodologies to support the evolving demands of the financial sector.

Deep Transfer Learning for Classification of Late Gadolinium Enhancement Cardiac MRI Images into Myocardial Infarction, Myocarditis, and Healthy Classes: Comparison with Subjective Visual Evaluation.

Background/Objectives: To develop a computer-aided diagnosis (CAD) method for the classification of late gadolinium enhancement (LGE) cardiac MRI images into myocardial infarction (MI), myocarditis, and healthy classes using a fine-tuned VGG16 model hybridized with multi-layer perceptron (MLP) (VGG16-MLP) and assess our model's performance in comparison to various pre-trained base models and MRI readers. Methods: This study included 361 LGE images for MI, 222 for myocarditis, and 254 for the healthy class. The left ventricle was extracted automatically using a U-net segmentation model on LGE images. Fine-tuned VGG16 was performed for feature extraction. A spatial attention mechanism was implemented as a part of the neural network architecture. The MLP architecture was used for the classification. The evaluation metrics were calculated using a separate test set. To compare the VGG16 model's performance in feature extraction, various pre-trained base models were evaluated: VGG19, DenseNet121, DenseNet201, MobileNet, InceptionV3, and InceptionResNetV2. The Support Vector Machine (SVM) classifier was evaluated and compared to MLP for the classification task. The performance of the VGG16-MLP model was compared with a subjective visual analysis conducted by two blinded independent readers. Results: The VGG16-MLP model allowed high-performance differentiation between MI, myocarditis, and healthy LGE cardiac MRI images. It outperformed the other tested models with 96% accuracy, 97% precision, 96% sensitivity, and 96% F1-score. Our model surpassed the accuracy of Reader 1 by 27% and Reader 2 by 17%. Conclusions: Our study demonstrated that the VGG16-MLP model permits accurate classification of MI, myocarditis, and healthy LGE cardiac MRI images and could be considered a reliable computer-aided diagnosis approach specifically for radiologists with limited experience in cardiovascular imaging.

Adapting to evolving MRI data: A transfer learning approach for Alzheimer's disease prediction.

Integrating 3D magnetic resonance imaging (MRI) with machine learning has shown promising results in healthcare, especially in detecting Alzheimer's Disease (AD). However, changes in MRI technologies and acquisition protocols often yield limited data, leading to potential overfitting. This study explores Transfer Learning (TL) approaches to enhance AD diagnosis using a Baseline model consisting of a 3D-Convolutional Neural Network trained on 80 3T MRI scans. Two scenarios are explored: (A) utilizing historical data to address changes in MRI acquisitions (from 1.5T to 3T MRI), and (B) adapting 2D models pre-trained on ImageNet (ResNet18, ResNet50, ResNet101) for 3D image processing when historical data is unavailable. In both scenarios, two modeling approaches are tested. The General Approach involves distinct feature extraction and classification steps, using Radiomic features and TL-based features evaluated with six classifiers. The Deep Approach integrates these steps by fine-tuning the pre-trained models for AD diagnosis. In scenario (A), TL significantly boosts the Baseline's accuracy from 63% to 99%. In scenario (B), Radiomic features better represents 3D MRI than TL-features in the General Approach. Nonetheless, fine-tuning models pre-trained on natural images can increase the Baseline's accuracy by up to 12 percentage points, achieving an overall accuracy of 83%.

Development and evaluation of a 4M taxonomy from nursing home staff text messages using a fine-tuned generative language model.

This study aimed to explore the utilization of a fine-tuned language model to extract expressions related to the Age-Friendly Health Systems 4M Framework (What Matters, Medication, Mentation, and Mobility) from nursing home worker text messages, deploy automated mapping of these expressions to a taxonomy, and explore the created expressions and relationships. The dataset included 21357 text messages from healthcare workers in 12 Missouri nursing homes. A sample of 860 messages was annotated by clinical experts to form a "Gold Standard" dataset. Model performance was evaluated using classification metrics including Cohen's Kappa (κ), with κ ≥ 0.60 as the performance threshold. The selected model was fine-tuned. Extractions were clustered, labeled, and arranged into a structured taxonomy for exploration. The fine-tuned model demonstrated improved extraction of 4M content (κ = 0.73). Extractions were clustered and labeled, revealing large groups of expressions related to care preferences, medication adjustments, cognitive changes, and mobility issues. The preliminary development of the 4M model and 4M taxonomy enables knowledge extraction from clinical text messages and aids future development of a 4M ontology. Results compliment themes and findings in other 4M research. This research underscores the need for consensus building in ontology creation and the role of language models in developing ontologies, while acknowledging their limitations in logical reasoning and ontological commitments. Further development and context expansion with expert involvement of a 4M ontology are necessary.

SVDDD: SAR Vehicle Target Detection Dataset Augmentation Based on Diffusion Model

In the field of target detection using synthetic aperture radar (SAR) images, deep learning-based supervised learning methods have demonstrated outstanding performance. However, the effectiveness of deep learning methods is largely influenced by the quantity and diversity of samples in the dataset. Unfortunately, due to various constraints, the availability of labeled image data for training SAR vehicle detection networks is quite limited. This scarcity of data has become one of the main obstacles hindering the further development of SAR vehicle detection. In response to this issue, this paper collects SAR images of the Ka, Ku, and X bands to construct a labeled dataset for training Stable Diffusion and then propose a framework for data augmentation for SAR vehicle detection based on the Diffusion model, which consists of a fine-tuned Stable Diffusion model, a ControlNet, and a series of methods for processing and filtering images based on image clarity, histogram, and an influence function to enhance the diversity of the original dataset, thereby improving the performance of deep learning detection models. In the experiment, the samples we generated and screened achieved an average improvement of 2.32%, with a maximum of 6.6% in mAP75 on five different strong baseline detectors.

The Spillover Effects of Cryptocurrency Trading Ban and Social Media on Domestic Fintech

In 2021, China implemented a sweeping cryptocurrency trading ban, significantly disrupting the cryptocurrency market. However, the impact of this regulatory action on domestic fintech companies has largely been overlooked. This paper investigates two key research questions: first, whether the 2021 Chinese cryptocurrency trading ban had spillover effects on the domestic fintech sector, and second, whether crypto-related social media sentiment influenced fintech companies differently before and after the ban. To address these questions, we employ a GARCH(1,1) model to analyze market volatility and trends. Additionally, we train a fine-tuned BERT model to enhance the accuracy of social media sentiment analysis. Our findings reveal that the cryptocurrency ban not only directly impacted the crypto market but also led to significant negative spillover effects on fintech firms. Following the ban, social media sentimentespecially concerning Bitcoinbecame a dominant factor, with fintech companies serving as a proxy for exposure to the digital finance sector. Positive sentiment surrounding cryptocurrencies served as a critical signal for speculative interest in fintech stocks post-ban.

Natural language processing to evaluate texting conversations between patients and healthcare providers during COVID-19 Home-Based Care in Rwanda at scale.

Community isolation of patients with communicable infectious diseases limits spread of pathogens but our understanding of isolated patients' needs and challenges is incomplete. Rwanda deployed a digital health service nationally to assist public health clinicians to remotely monitor and support SARS-CoV-2 cases via their mobile phones using daily interactive short message service (SMS) check-ins. We aimed to assess the texting patterns and communicated topics to better understand patient experiences. We extracted data on all COVID-19 cases and exposed contacts who were enrolled in the WelTel text messaging program between March 18, 2020, and March 31, 2022, and linked demographic and clinical data from the national COVID-19 registry. A sample of the text conversation corpus was English-translated and labeled with topics of interest defined by medical experts. Multiple natural language processing (NLP) topic classification models were trained and compared using F1 scores. Best performing models were applied to classify unlabeled conversations. Total 33,081 isolated patients (mean age 33·9, range 0-100), 44% female, including 30,398 cases and 2,683 contacts) were registered in WelTel. Registered patients generated 12,119 interactive text conversations in Kinyarwanda (n = 8,183, 67%), English (n = 3,069, 25%) and other languages. Sufficiently trained large language models (LLMs) were unavailable for Kinyarwanda. Traditional machine learning (ML) models outperformed fine-tuned transformer architecture language models on the native untranslated language corpus, however, the reverse was observed of models trained on English-only data. The most frequently identified topics discussed included symptoms (69%), diagnostics (38%), social issues (19%), prevention (18%), healthcare logistics (16%), and treatment (8·5%). Education, advice, and triage on these topics were provided to patients. Interactive text messaging can be used to remotely support isolated patients in pandemics at scale. NLP can help evaluate the medical and social factors that affect isolated patients which could ultimately inform precision public health responses to future pandemics.

Examination Process Modeling for Intelligent Patent Management: A Multi-aspect Neural Sequential Approach

Recently, the booming growth of patent applications has brought an unprecedented challenge in performing efficient intellectual property management. Therefore, intelligent approaches are urgently needed to analyze intrinsic patterns of patents. However, a long-standing obstacle is the lack of effective methods for modeling the dynamic and diverse examination process of patent applications, which can benefit a wide range of downstream tasks for patent management. In fact, the major challenges lie in how to discover and integrate domain-specific properties from large-scale unlabeled examination data. To this end, in this paper, we propose a Self-supervised Examination Process Modeling (SEPM) framework to learn the contextualized embedding for patents through modeling their examination processes. Specifically, we first design a multi-aspect event embedding layer, which leverages the fine-tuned language model, frequent-pattern embedding, and time encoding to capture the semantic, frequent-pattern and temporal information of examination events, respectively. Then, a mutual-information-aware integration layer is applied to fuse the extracted features into multi-aspect embedding considering their mutual interactions. Further, we develop a multi-objective sequential neural network for learning the contextualized patent representation, which is achieved through jointly learning two self-supervised objectives, namely event code and event lag auto-regression. To explore the application potential of SEPM, we fine-tune the well-trained model for three important downstream tasks of patent management, including the prediction of next events, patent classification, and grant prediction. In the end, extensive experiments with real-world data from the US Patent and Trademark Office verify the effectiveness and application prospects of the proposed framework.

Fine-tuning a local LLaMA-3 large language model for automated privacy-preserving physician letter generation in radiation oncology.

Generating physician letters is a time-consuming task in daily clinical practice. This study investigates local fine-tuning of large language models (LLMs), specifically LLaMA models, for physician letter generation in a privacy-preserving manner within the field of radiation oncology. Our findings demonstrate that base LLaMA models, without fine-tuning, are inadequate for effectively generating physician letters. The QLoRA algorithm provides an efficient method for local intra-institutional fine-tuning of LLMs with limited computational resources (i.e., a single 48 GB GPU workstation within the hospital). The fine-tuned LLM successfully learns radiation oncology-specific information and generates physician letters in an institution-specific style. ROUGE scores of the generated summary reports highlight the superiority of the 8B LLaMA-3 model over the 13B LLaMA-2 model. Further multidimensional physician evaluations of 10 cases reveal that, although the fine-tuned LLaMA-3 model has limited capacity to generate content beyond the provided input data, it successfully generates salutations, diagnoses and treatment histories, recommendations for further treatment, and planned schedules. Overall, clinical benefit was rated highly by the clinical experts (average score of 3.4 on a 4-point scale). With careful physician review and correction, automated LLM-based physician letter generation has significant practical value.

Deep-Learning-based Automated Identification of Ventriculoperitoneal-Shunt Valve Models from Skull X-rays.

Ventriculoperitoneal shunts (VPS) are an essential part of the treatment of hydrocephalus, with numerous valve models available with different ways of indicating pressure levels. The model types often need to be identified on X‑rays to assess pressure levels using amatching template. Artificial intelligence (AI), in particular deep learning, is ideally suited to automate repetitive tasks such as identifying different VPS valve models. The aim of this work was to investigate whether AI, in particular deep learning, allows the identification of VPS models in cranial X‑rays. 959 cranial X‑rays of patients with aVPS were included and reviewed for image quality and complete visualization of VPS valves. The images included four VPS model types: Codman Hakim (n = 774, 81%), Codman Certas Plus (n = 117, 12%), Sophysa Sophy Mini SM8 (n = 35, 4%) and proGAV 2.0 (n = 33, 3%). AConvolutional Neural Network (CNN) was trained using stratified five-fold cross-validation to classify the four VPS model types in the dataset. Afinetuned CNN pretrained on the ImageNet dataset as well as amodel trained from scratch were compared. The averaged performance and uncertainty metrics were evaluated across the cross-validation splits. The fine-tuned model identified VPS valve models with amean accuracy of 0.98 ± 0.01, macro-averaged F1 score of 0.93 ± 0.04, arecall of 0.94 ± 0.03 and aprecision of 0.95 ± 0.08 across the five cross-validation splits. Automatic classification of VPS valve models in skull X‑rays, using fully automatable preprocessing steps and aCNN, is feasible. This is an encouraging finding to further explore the possibility of automating VPS valve model identification and pressure level reading in skull X‑rays.

On Assessing the Performance of LLMs for Target-Level Sentiment Analysis in Financial News Headlines

The importance of sentiment analysis in the rapidly evolving financial markets is widely recognized for its ability to interpret market trends and inform investment decisions. This study delves into the target-level financial sentiment analysis (TLFSA) of news headlines related to stock. The study compares the performance in the TLFSA task of various sentiment analysis techniques, including rule-based models (VADER), fine-tuned transformer-based models (DistilFinRoBERTa and Deberta-v3-base-absa-v1.1) as well as zero-shot large language models (ChatGPT and Gemini). The dataset utilized for this analysis, a novel contribution of this research, comprises 1476 manually annotated Bloomberg headlines and is made publicly available (due to copyright restrictions, only the URLs of Bloomberg headlines with the manual annotations are provided; however, these URLs can be used with a Bloomberg terminal to reconstruct the complete dataset) to encourage future research on this subject. The results indicate that the fine-tuned Deberta-v3-base-absa-v1.1 model performs better across all evaluation metrics than other evaluated models in TLFSA. However, LLMs such as ChatGPT-4, ChatGPT-4o, and Gemini 1.5 Pro provide similar performance levels without the need for task-specific fine-tuning or additional training. The study contributes to assessing the performance of LLMs for financial sentiment analysis, providing useful insights into their possible application in the financial domain.

Using a machine learning framework for natural language processing to create a high-resolution carbon emission map for urban manufacturing

Managing carbon emissions from the manufacturing sector is crucial for sustainable development, and effective identification of manufacturing land is key to achieving this goal. However, current methods for identifying urban manufacturing land remain inadequate. In this study, we employ a fine-tuned, pre-trained natural language processing model based on Bidirectional Encoder Representations from Transformers to classify points of interest data into manufacturing industry categories. This approach enables us to identify manufacturing land and allocate corresponding carbon emissions data to specific parcels. The global Moran’s Index and local Moran’s Index are applied to analyze the relationship between manufacturing concentration and carbon emission intensity. The results demonstrate that the fine-tuned model achieved an accuracy rate of 91.6% on the test set, successfully identifying 98.72% of the manufacturing land in the study area. The intensity of carbon emissions from manufacturing exhibits a significant positive spatial correlation, with urban areas characterized by high-high and low-low clustering of emissions. In rural areas, high-emission manufacturers tend to be co-located with low-emission enterprises. Within individual manufacturing sectors, most exhibit low-low clustering, suggesting a potential relationship between such clustering and lower carbon emissions. This study provides detailed spatial data for the management of carbon emissions in the manufacturing sector and addresses the gap in micro-scale research on the correlation between manufacturing concentration and carbon emissions.

Real-time human–computer interface based on eye gaze estimation from low-quality webcam images: integration of convolutional neural networks, calibration, and transfer learning

Abstract Eye gaze estimation represents a well-established research domain within computer vision. It has a wide range of practical applications in numerous fields, including human–computer interaction (HCI) for cursor control, health care, and virtual reality, enhancing its suitability for adoption throughout the scientific community. Different methods have been used for eye gaze estimation, such as model based, feature based, and appearance based. The appearance-based method is mainly used because it directly estimates an individual’s gaze direction from images/videos rather than depending on specific features or geometric models. This article developed an appearance-based, real-time generic eye gaze system for HCI to control the cursor through the eye using the convolutional neural network (CNN), calibration, and transfer learning. The study employed low-quality eye images captured from a conventional desktop webcam, enabling the proposed methodology to be implemented on any computer system equipped with a similar web camera without the need for supplementary hardware. Initially, the labeled dataset of both eyes is collected using the webcam. Then, a CNN model is trained by inputting left and right eye images to predict the gaze coordinate as output. We applied the calibration and transfer learning approach to the trained models to make a generic model for new users. In real-time use, the first step is calibration, where the user’s eye images are captured for various screen coordinates, and transfer learning is employed to fine-tune the pre-trained model according to the user’s eyes. Then, the fine-tuned model is used for eye gaze prediction to control the cursor. The system’s performance is evaluated using a test group of multiple users, and it demonstrated an average visual angle accuracy of 2.08 degrees before calibration, which notably improved to 1.81 degrees after the calibration process.

A scientific-article key-insight extraction system based on multi-actor of fine-tuned open-source large language models

The exponential growth of scientific articles has presented challenges in information organization and extraction. Automation is urgently needed to streamline literature reviews and enhance insight extraction. We explore the potential of Large Language Models (LLMs) in key-insights extraction from scientific articles, including OpenAI’s GPT-4.0, MistralAI’s Mixtral 8 × 7B, 01AI’s Yi, and InternLM’s InternLM2. We have developed an article-level key-insight extraction system based on LLMs, calling it ArticleLLM. After evaluating the LLMs against manual benchmarks, we have enhanced their performance through fine-tuning. We propose a multi-actor LLM approach, merging the strengths of multiple fine-tuned LLMs to improve overall key-insight extraction performance. This work demonstrates not only the feasibility of LLMs in key-insight extraction, but also the effectiveness of cooperation of multiple fine-tuned LLMs, leading to efficient academic literature survey and knowledge discovery.