Improving Code Summarization Through Automated Quality Assurance

Abstract

The code summarization task aims to generate brief descriptions of source code automatically. It is beneficial for developers to understand source code. However, almost all of current code summarization approaches may generate low-quality (BLEU4<40) summaries, which will mislead developers. Previous work has shown that it is possible to conduct quality assurance for document generation (QA4DG) and improve the practicability of document generation approaches. Code summarization can also be regarded as a document generation task. This work aims to investigate whether QA4DG approaches can be leveraged to improve code summarization. Specifically, we first investigate whether existing QA4DG approaches can be plugged in code summarization approaches. We find that an automated quality assurance framework for commit message generation named QACom performs best. In-spired by the idea behind QAcom, we propose an ensemble code summarization approach called Ensum. Precisely, given a code snippet, Ensum first uses current code summarization approaches to generate candidate summaries. Then, Ensum predicts the quality of each candidate summary using a collaborative filtering-based component and a retrieval-based component and selects the best candidate summary as the output. Experimental results on two public datasets show that Ensum outperforms three state-of-the-art single approaches and one ensemble approach for code summarization in terms of BLEU-4, METEOR, and ROUGE-L.