Neural Knowledge Base Research Articles

Let’s Discover More API Relations: A Large Language Model-based AI Chain for Unsupervised API Relation Inference

APIs have intricate relations that can be described in text and represented as knowledge graphs to aid software engineering tasks. Existing relation extraction methods have limitations, such as limited API text corpus and affected by the characteristics of the input text. To address these limitations, we propose utilizing large language models (LLMs) (e.g., gpt-3.5) as a neural knowledge base for API relation inference. This approach leverages the entire Web used to pre-train LLMs as a knowledge base and is insensitive to the context and complexity of input texts. To ensure accurate inference, we design an AI chain consisting of three AI modules: API Fully Qualified Name (FQN) Parser, API Knowledge Extractor, and API Relation Decider. The accuracy of the API FQN Parser and API Relation Decider is 0.81 and 0.83, respectively. Using the generative capacity of the LLM and our approach’s inference capability, we achieve an average F1 value of 0.76 under the three datasets, significantly higher than the state-of-the-art method’s average F1 value of 0.40. Compared to the original CoT and modularized CoT methods, our AI chain design has improved the performance of API relation inference by 71% and 49%, respectively. Meanwhile, the prompt ensembling strategy enhances the performance of our approach by 32%. The API relations inferred by our method can be further organized into structured forms to provide support for other software engineering tasks.

FQN Inference in Partial Code by Prompt-tuned Language Model of Code

Partial code usually involves non-fully-qualified type names (non-FQNs) and undeclared receiving objects. Resolving the FQNs of these non-FQN types and undeclared receiving objects (referred to as type inference) is the prerequisite to effective search and reuse of partial code. Existing dictionary-lookup based methods build a symbolic knowledge base of API names and code contexts, which involve significant compilation overhead and are sensitive to unseen API names and code context variations. In this article, we propose using a p rompt-tuned c o de m asked language mod e l (MLM) as a neural knowledge base for type inference, called POME, which is lightweight and has minimal requirements on code compilation. Unlike the existing symbol name and context matching for type inference, POME infers the FQNs syntax and usage knowledge encapsulated in prompt-tuned code MLM through a colze-style fill-in-blank strategy. POME is integrated as a plug-in into web and integrated development environments (IDE) to assist developers in inferring FQNs in the real world. We systematically evaluate POME on a large amount of source code from GitHub and Stack Overflow, and explore its generalization and hybrid capability. The results validate the effectiveness of the POME design and its applicability for partial code type inference, and they can be easily extended to different programming languages (PL). POME can also be used to generate a PL-hybrid type inference model for providing a one-for-all solution. As the first of its kind, our neural type inference method opens the door to many innovative ways of using partial code.