Automated Program Generation Research Articles

On the Reliability and Explainability of Language Models for Program Generation

Recent studies have adopted pre-trained language models, such as CodeT5 and CodeGPT, for automated program generation tasks like code generation, repair, and translation. Numerous language model based approaches have been proposed and evaluated on various benchmark datasets, demonstrating promising performance. However, there is still uncertainty about the reliability of these models, particularly their realistic ability to consistently transform code sequences. This raises a question: are these techniques sufficiently trustworthy for automated program generation? Consequently, further research is needed to understand model logic and assess reliability and explainability. To bridge these research gaps, we conduct a thorough empirical study of eight popular language models on five representative datasets to determine the capabilities and limitations of automated program generation approaches. We further employ advanced explainable AI approaches to highlight the tokens that significantly contribute to the code transformation. We discover that state-of-the-art approaches suffer from inappropriate performance evaluation stemming from severe data duplication, causing overoptimistic results. Our explainability analysis reveals that, in various experimental scenarios, language models can recognize code grammar and structural information, but they exhibit limited robustness to changes in input sequences. Overall, more rigorous evaluation approaches and benchmarks are critical to enhance the reliability and explainability of automated program generation moving forward. Our findings provide important guidelines for this goal.

A Programming Paradigm Combining Programmer and Neural Network to Promote Automated Program Generation

A Programming Paradigm Combining Programmer and Neural Network to Promote Automated Program Generation

An online robot trajectory planning and programming support system for industrial use

The manufacturing industry today is still looking for enhancement of their production. Programming of articulated production robots is a major area for improvement. Today, offline simulation modified by manual programming is widely used to reduce production downtimes but requires financial investments in terms of additional personnel and equipment costs. The requirements have been evaluated considering modern manufacturing aspects and a new online robot trajectory planning and programming support system is presented for industrial use. The proposed methodology is executed solely online, rendering offline simulation obsolete and thereby reduces costs. To enable this system, a new cell-based Voronoi generation algorithm, together with a trajectory planner, is introduced. The robot trajectories so achieved are comparable to manually programmed robot programs. The results for a Mitsubishi RV-2AJ five axis industrial robot are presented. Highlights? Evaluation of modern manufacturing aspects. ? Supported programming of articulated production robots. ? Path planning for articulated robots. ? Automated program generation of maintainable robot programs. ? Enhancement of manufacture industry production.

事例ベース推論を用いた画像処理プログラム自動合成システム

事例ベース推論を用いた画像処理プログラム自動合成システム

Automated software synthesis: an application in mechanical CAD

Automated program synthesis has not gained widespread acceptance among software practitioners despite considerable efforts by several researchers. We outline some of the difficulties in applying program synthesis for practical problems and argue that a careful analysis of the cost vs. benefit tradeoff is essential when considering such an approach. We describe a successful application of automated program generation for synthesizing geometric constraint satisfaction routines in the domain of mechanical CAD. We present a general framework for modeling and solving the problem, illustrate the framework using examples from the geometric constraint satisfaction domain, and describe experimental results on productivity increase using this approach. We also discuss characteristics of the problem domain and our approach that were critical for success.

A statistical expert system for simulation analysis

Efforts on a number of fronts are being made to enhance the usefulness of simulation as a tool for analysis. Graphical input, animation, automated program generation, model management utilities, and user interactive environments all address the issue of ease of use. Even with the current emphasis in this regard, simulation suffers from several shortcomings which preclude even greater application of the methodology. One of the more signifi cant limitations is the dependence of simulation on statistical methodologies. Practically every phase of simulation analysis in volves one or more statistical issues. And the fact of the matter is that many simulation practitioners have neither the statistical skills nor the access to qualified statisticians to effectively address these crucial issues. The obvious solution to this problem would be to have an ex pert statistician available during every simulation study to answer any questions concerning statistical issues. A more practical alter native is to make available an expert system which could func tion as a statistical advisor to simulation analysts. This option seems ideally suited for expert system techniques since it is well- defined and narrowly bounded and an appropriate body of statistical knowledge exists. The objective of this research was to develop such an expert statistical consultant. The system would be able to advise as well as perform appropriate computations for a variety of statistical issues including input data analysis, design of experiments, and output analysis. An overview of the proposed system is given and illustrations of the capabilities of the system are presented.