Evaluating and improving LLM-based competitive program generation

In recent years, tremendous strides have been made in the area of program synthesis due to the leveraging of highly parameterized transformer models. Such models have demonstrated near human levels of performance on tasks such as bug detection, computer language translation, and competitive programming. Unfortunately, little research has been done in the exploration of decoding methodologies for such models, despite the semantic and structural differences between human and programming languages. In this paper, we propose extensions to commonly used decoding strategies, which incorporate additional constraints on non-concise and inefficient program generations. Our approaches have shown comparable performance on program generation tasks while producing programs requiring fewer lines of code and a reduced number of looping operations on average compared to traditional methods of decoding.

This article discusses key approaches to the formation of the competitiveness of higher education programs under modern conditions. The author presents and substantiates the problems of insufficient attractiveness of programs for the modern generation of applicants (generation Z), analyzes a possible list of indicators for assessing the level of competitiveness of education programs, and formulates the main principles for creating a competitive educational program: integration, reliance on the university's key competencies, monitoring of industry and labor market issues in real time, openness, and efficiency.

Automated program generation (APG) is a concept of automatically making a computer program. Toward this goal, transferring automated program repair (APR) to APG can be considered. APR modifies the buggy input source code to pass all test cases. APG regards empty source code as initially failing all test cases, i.e., containing multiple bugs. Search-based APR repeatedly generates program variants and evaluates them. Many traditional APR systems evaluate the fitness of variants based on the number of passing test cases. However, when source code contains multiple bugs, this fitness function lacks the expressive power of variants. In this paper, we propose the application of a multi-objective genetic algorithm to APR in order to improve efficiency. We also propose a new crossover method that combines two variants with complementary test results, taking advantage of the high expressive power of multi-objective genetic algorithms for evaluation. We tested the effectiveness of the proposed method on competitive programming tasks. The obtained results showed significant differences in the number of successful trials and the required generation time.