Recent studies have highlighted the potential of Large Language Models (LLMs) to become experts in specific areas of knowledge through the utilization of techniques that enhance their context. Nevertheless, an interesting and underexplored application in the literature is the creation of an LLM that specializes in research projects, as it could streamline the process of project ideation and accelerate the advancement of research initiatives. In this regard, the aim of this work is to develop a tool based on LLM technology capable of assisting the employees of technology centers in answering their queries related to research projects funded under the Horizon 2020 program. By facilitating the identification of suitable funding calls and the formation of consortia with partners meeting specific requirements, tasks that are traditionally time-intensive, the proposed tool has the potential to improve operational efficiency and enable technology centers to allocate their resources more effectively. To improve the model’s baseline performance, context extension techniques such as Retrieved Augmented Generation (RAG) and prompt engineering were explored. Specifically, different RAG approaches and configurations, along with a specialized prompt, were tested on the LLaMA 3 70B model, and their results were compared to those obtained without context extension. The proposed evaluation metrics, which aligned with human judgment while maintaining objectivity, revealed that RAG systems outperformed the standalone LLaMA 3 70B, achieving a rate of optimal responses of up to 46% compared to 0% for the baseline model. These findings emphasize that integrating RAG and prompt engineering pipelines into LLMs can address key limitations, such as generating accurate and informative answers. Moreover, this study demonstrates the practical feasibility of leveraging advanced LLM configurations to support research-driven organizations, highlighting a pathway for the further development of intelligent tools that enhance productivity and foster innovation in the research domain.
Read full abstract