Modern IDEs Research Articles

Rethinking AI code generation: a one-shot correction approach based on user feedback

Code generation has become an integral feature of modern IDEs, gathering significant attention. Notable approaches like GitHub Copilot and TabNine have been proposed to tackle this task. However, these tools may shift code writing tasks towards code reviewing, which involves modification from users. Despite the advantages of user feedback, their responses remain transient and lack persistence across interaction sessions. This is attributed to the inherent characteristics of generative AI models, which require explicit re-training for new data integration. Additionally, the non-deterministic and unpredictable nature of AI-powered models limits thorough examination of their unforeseen behaviors. We propose a methodology named One-shot Correction to mitigate these issues in natural language to code translation models with no additional re-training. We utilize decomposition techniques to break down code translation into sub-problems. The final code is constructed using code snippets of each query chunk, extracted from user feedback or selectively generated from a generative model. Our evaluation indicates comparable or improved performance compared to other models. Moreover, the methodology offers straightforward and interpretable approaches, which enable in-depth examination of unexpected results and facilitate insights for potential enhancements. We also illustrate that user feedback can substantially improve code translation models without re-training. Ultimately, we develop a preliminary GUI application to demonstrate the utility of our methodology in simplifying customization and assessment of suggested code for users.

A new architecture reconciling refactorings and transformations

Refactorings are behavior-preserving code transformations. They are a recommended software development practice and are now a standard feature in modern IDEs. There are however many situations where developers need to perform mere transformations (non-behavior-preserving) or to mix refactorings and transformations. Little work exists on the analysis of transformations implementation, how refactorings could be composed of smaller, reusable, parts (simple transformations or other refactorings), and, conversely, how transformations could be reused in isolation or to compose new refactorings. In a previous article, we started to analyze the seminal implementation of refactorings as proposed in the Ph.D. of D. Roberts, and whose evolution is available in the Pharo IDE. We identified a dichotomy between the class hierarchy of refactorings (56 classes) and that of transformations (70 classes). We also noted that there are different kinds of preconditions for different purposes (applicability preconditions or behavior-preserving preconditions). In this article, we go further by proposing a new architecture that: (i) supports two important scenarios (interactive use or scripting, i.e., batch use); (ii) defines a clear API unifying refactorings and transformations; (iii) expresses refactorings as decorators over transformations, and; (iv) formalizes the uses of the different kinds of preconditions, thus supporting better user feedback. We are in the process of migrating the existing Pharo refactorings to this new architecture. Current results show that elementary transformations such as the Add Method transformation is reused in 24 refactorings and 11 other transformations; and the Remove Method transformation is reused in 11 refactorings and 7 other transformations.

Teaching language processing with the PAMOJA framework

Existing educational tools for language processors have varying capabilities. There is no single tool that covers every aspect of language processors. As a result, there is a possibility that the educator and students will need multiple tools, switching between different specification notations, organization and interpretation of outputs, which could result in a steep learning curve. PAMOJA is a Java-based component framework with a broader scope — it supports construction of grammar-aware applications using the rapid application development style found in modern IDEs such as NetBeans. This paper investigated the possibility of using PAMOJA to develop educational tools for language processing courses. We conducted two major studies. The first study, summarized six design considerations, identified in the literature, for language processor educational tools. Then, we identified three example applications and demonstrated how to construct tools for them using PAMOJA. These include building and visualizing lexical scanners and parsers, as well as constructing a front-end for a software language, using a subset of Java language as an example. The second study evaluated the PAMOJA approach in relation to the identified design considerations, and by analyzing student and educator perceptions. The results seem to demonstrate a positive reception and acceptance, concluding that its application would facilitate the design of language processor teaching tools and learning language processors.

Heuristic and Neural Network Based Prediction of Project-Specific API Member Access

Code completion is to predict the rest of a statement a developer is typing. Although advanced code completion approaches have greatly improved the accuracy of code completion in modern IDEs, it remains challenging to predict project-specific API method invocations or field accesses because little knowledge about such elements could be learned in advance. To this end, in this paper we propose an accurate approach called HeeNAMA to suggesting the next project-specific API member access. HeeNAMA focuses on a specific but common case of code completion: suggesting the following member access whenever a project-specific API instance is followed by a dot on the right hand side of an assignment. By focusing on such a specific case, HeeNAMA can take full advantages of the context of the code completion, including the type of the left hand side expression of the assignment, the identifier on the left hand side, the type of the base instance, and similar assignments typed in before. All such information together enables highly accurate code completion. Given an incomplete assignment, HeeNAMA generates the initial candidate set according to the type of the base instance, and excludes those candidates that are not type compatible with the left hand side of the assignment. If the enclosing project contains assignments highly similar to the incomplete assignment, it makes suggestions based on such assignments. Otherwise, it selects the one from the initial candidate set that has the greatest lexical similarity with the left hand side of the assignment. Finally, it employs a neural network to filter out risky predictions, which guarantees high precision. Evaluation results on open-source applications suggest that compared to the state-of-the-art approaches and the state-of-the-practice tools HeeNAMA improves precision and recall by 70.68% and 25.23%, relatively.

An empirical investigation of the effects of type systems and code completion on API usability using TypeScript and JavaScript in MS visual studio

Recent empirical studies that compared static and dynamic type systems on API usability showed a positive impact of static type systems on developer productivity in most cases. Nevertheless, it is unclear how large this effect is in comparison to other factors. One obvious factor in programming is tooling: It is commonly accepted that modern IDEs have a large positive impact on developers, although it is not clear which parts of modern IDEs are responsible for that. One possible---and for most developers obvious candidate---is code completion. This paper describes a 2x2 randomized trial that compares JavaScript and Microsoft's statically typed alternative TypeScript with and without code completion in MS Visual Studio. While the experiment shows (in correspondence to previous experiments) a large positive effect of the statically typed language TypeScript, the code completion effect is not only marginal, but also just approaching statistical significance. This seems to be an indicator that the effect of static type systems is larger than often assumed, at least in comparison to code completion.

Equipping IDEs with XML-Path Reasoning Capabilities

One of the challenges in Web development is to achieve a good level of quality in terms of code size and runtime performance for popular domain-specific languages such as XQuery, XSLT, and XML Schema. We present the first IDE augmented with static detection of inconsistent XPath expressions that assists the programmer with simplifying development and debugging of any application involving XPath expressions. The tool is based on newly developed formal verification techniques based on expressive modal logics, which are now mature enough to be introduced in the process of software development. We further develop this idea in the context of XQuery for which we introduce an analysis for identifying and eliminating dead code automatically. This proof of concept aims at illustrating the benefits of equipping modern IDEs with reasoning capabilities.

Alias calculus, change calculus and frame inference

Alias analysis, which determines whether two expressions in a program may reference to the same object, has many potential applications in program construction and verification. We have developed a theory for alias analysis, the “alias calculus”, implemented its application to an object-oriented language, and integrated the result into a modern IDE. The calculus has a higher level of precision than many existing alias analysis techniques.One of the principal applications is to allow automatic change analysis, which leads to inferring “modifies” clauses, providing a significant advance towards addressing the Frame Problem. Experiments were able to infer the “modifies” clauses of an existing formally specified library. Other applications, in particular to concurrent programming, also appear possible.The article presents the calculus, the application to frame inference including experimental results, and other projected applications. The ongoing work includes building more efficient model capturing aliasing properties and soundness proof for its essential elements.

Semi-automatic rename refactoring for JavaScript

Modern IDEs support automated refactoring for many programming languages, but support for JavaScript is still primitive. To perform renaming, which is one of the fundamental refactorings, there is often no practical alternative to simple syntactic search-and-replace. Although more sophisticated alternatives have been developed, they are limited by whole-program assumptions and poor scalability. We propose a technique for semi-automatic refactoring for JavaScript, with a focus on renaming. Unlike traditional refactoring algorithms, semi-automatic refactoring works by a combination of static analysis and interaction with the programmer. With this pragmatic approach, we can provide scalable and effective refactoring support for real-world code, including libraries and incomplete applications. Through a series of experiments that estimate how much manual effort our technique demands from the programmer, we show that our approach is a useful improvement compared to search-and-replace tools.

Cloud refactoring: automated transitioning to cloud-based services

Using cloud-based services can improve the performance, reliability, and scalability of a software application. However, transitioning an application to use cloud-based services is difficult, costly, and error-prone. The required re-engineering effort includes migrating to the cloud the functionality to be accessed as remote cloud-based services and re-targeting the client code accordingly. In addition, the client must be able to detect and handle the faults raised in the process of invoking the services. As a means of streamlining this transitioning, we developed a set of refactoring techniques--automated, IDE-assisted program transformations that eliminate the need to change programs by hand. In particular, we show how a programmer can extract services, add fault tolerance functionality, and adapt client code to invoke cloud services via refactorings integrated with a modern IDE. As a validation, we have applied our approach to automatically transform two third-party Java applications to use cloud-based services. We have also applied our approach to re-engineer a suite of services operated by General Electric to use cloud-based resources to better satisfy the GE business requirements.

Evaluating a query framework for software evolution data

With the steady advances in tooling to support software engineering, mastering all the features of modern IDEs, version control systems, and project trackers is becoming increasingly difficult. Answering even the most common developer questions can be surprisingly tedious and difficult. In this article we present a user study with 35 subjects to evaluate our quasi-natural language interface that provides access to various facets of the evolution of a software system but requires almost zero learning effort. Our approach is tightly woven into the Eclipse IDE and allows developers to answer questions related to source code, development history, or bug and issue management. The results of our evaluation show that our query interface can outperform classical software engineering tools in terms of correctness, while yielding significant time savings to its users and greatly advancing the state of the art in terms of usability and learnability.

An abstract interpretation framework for refactoring with application to extract methods with contracts

Method extraction is a common refactoring feature provided by most modern IDEs. It replaces a user-selected piece of code with a call to an automatically generated method. We address the problem of automatically inferring contracts (precondition, postcondition) for the extracted method. We require the inferred contract: (a) to be valid for the extracted method (validity); (b) to guard the language and programmer assertions in the body of the extracted method by an opportune precondition (safety); (c) to preserve the proof of correctness of the original code when analyzing the new method separately (completeness); and (d) to be the most general possible (generality). These requirements rule out trivial solutions (e.g., inlining, projection, etc). We propose two theoretical solutions to the problem. The first one is simple and optimal. It is valid, safe, complete and general but unfortunately not effectively computable (except for unrealistic finiteness/decidability hypotheses). The second one is based on an iterative forward/backward method. We show it to be valid, safe, and, under reasonable assumptions, complete and general. We prove that the second solution subsumes the first. All justifications are provided with respect to a new, set-theoretic version of Hoare logic (hence without logic), and abstractions of Hoare logic, revisited to avoid surprisingly unsound inference rules. We have implemented the new algorithms on the top of two industrial-strength tools (CCCheck and the Microsoft Roslyn CTP). Our experience shows that the analysis is both fast enough to be used in an interactive environment and precise enough to generate good annotations.

Exploiting Dynamic Information in IDEs Improves Speed and Correctness of Software Maintenance Tasks

Modern IDEs such as Eclipse offer static views of the source code, but such views ignore information about the runtime behavior of software systems. Since typical object-oriented systems make heavy use of polymorphism and dynamic binding, static views will miss key information about the runtime architecture. In this paper, we present an approach to gather and integrate dynamic information in the Eclipse IDE with the goal of better supporting typical software maintenance activities. By means of a controlled experiment with 30 professional developers, we show that for typical software maintenance tasks, integrating dynamic information into the Eclipse IDE yields a significant 17.5 percent decrease of time spent while significantly increasing the correctness of the solutions by 33.5 percent. We also provide a comprehensive performance evaluation of our approach.

Identification and application of Extract Class refactorings in object-oriented systems

Refactoring is recognized as an essential practice in the context of evolutionary and agile software development. Recognizing the importance of the practice, modern IDEs provide some support for low-level refactorings. A notable exception in the list of supported refactorings is the “Extract Class” refactoring, which is conceived to simplify large, complex, unwieldy and less cohesive classes.In this work, we describe a method and a tool, implemented as an Eclipse plugin, designed to fulfill exactly this need. Our method involves three steps: (a) recognition of Extract Class opportunities, (b) ranking of the identified opportunities in terms of the improvement each one is anticipated to bring about to the system design, and (c) fully automated application of the refactoring chosen by the developer. The first step relies on an agglomerative clustering algorithm, which identifies cohesive sets of class members within the system classes. The second step relies on the Entity Placement metric as a measure of design quality. Through a set of experiments we have shown that the tool is able to identify and extract new classes that developers recognize as “coherent concepts” and improve the design quality of the underlying system.

Инкрементальный анализ исходного кода на языках C/C++

The article describes a way of implementing static code analysis for C/C++ languages that allows to reduce dramatically the time of the second and following analysis iterations. Main idea behind algorithm based on fact that programs on C/C++ languages widely use common parts of code placed inside header files with declarations and such header files included inside of several compilation units of projects. The ratio of the number of tokens in the source file to the number of tokens in the included headers is small. It is possible to reduce time of analysis using cache of compilation results for rarely changed part of compilation unit and use it in following iterations of analysis during work inside modern IDEs. The method is implemented in a commercial product for MS Visual Studio. The result of implementation described method is reduced more that in ten times time needed to analyze changed source code. Such performance improvement allows to re-analyze code on every changed symbol inside IDE and issue defects found in source code on-the-fly while developer creates program. This method is implemented inside of product Klocwork inSight 9.5

Tool-supported refactoring for JavaScript

Refactoring is a popular technique for improving the structure of existing programs while maintaining their behavior. For statically typed programming languages such as Java, a wide variety of refactorings have been described, and tool support for performing refactorings and ensuring their correctness is widely available in modern IDEs. For the JavaScript programming language, however, existing refactoring tools are less mature and often unable to ensure that program behavior is preserved. Refactoring algorithms that have been developed for statically typed languages are not applicable to JavaScript because of its dynamic nature. We propose a framework for specifying and implementing JavaScript refactorings based on pointer analysis. We describe novel refactorings motivated by best practice recommendations for JavaScript programming, and demonstrate how they can be described concisely in terms of queries provided by our framework. Experiments performed with a prototype implementation on a suite of existing applications show that our approach is well-suited for developing practical refactoring tools for JavaScript.

Programming assurance cases in Agda

Agda is a modern functional programming language equipped with an interactive proof assistant as its developing environment. Its features include dependent types, type universe, inductive and coinductive families of types, pattern matching, records, and nested parameterized modules. Based on the "propositions as types, proofs as programs" correspondence in Martin-Löf's Type Theory, Agda lets users to construct, verify, and execute a smooth mixture of programs and proofs. Using Agda is similar to using an editor in a modern IDE. Users have more direct control over how programs / proofs are written than in automation-oriented systems using command-scripts for proof construction. Agda thus encourages users to express their ideas with more sophisticated dependently typed programming and less logical proofs. Programming techniques for readability and maintainability now translate to techniques for writing verified documents for human communication. Agda has been developed at Chalmers University of Technology by Ulf Norell and others. A growing international community of developers and users applies it in research, education, and industry. At AIST in Japan, we aim to introduce its merits to construction, verification, maintenance, and run-time evaluation of "assurance cases", which are documented bodies of systems assurance arguments used as the hub for assurance- and risk-communication among stakeholders. The talk gives an overview of Agda and presents our current effort on programming assurance cases in Agda.

Specifying and implementing refactorings

Modern IDEs for object-oriented languages like Java provide support for a basic set of simple automated refactorings whose behaviour is easy to describe intuitively. It is, however, surprisingly difficult to specify their behaviour in detail. In particular, the popular precondition-based approach tends to produce somewhat unwieldy descriptions if advanced features of the object language are taken into account. This has resulted in refactoring implementations that are complex, hard to understand, and even harder to maintain, yet these implementations themselves are the only precise "specification" of many refactorings. We have in past work advocated a different approach based on several complementary notions of dependencies that guide the implementation, and on the concept of microrefactorings that structure it. We show in this work that these concepts are powerful enough to provide high-level specifications of many of the refactorings implemented in Eclipse. These specifications are precise enough to serve as the basis of a clean-room reimplementation of these refactorings that is very compact, yet matches Eclipse's for features and outperforms it in terms of correctness.

Refactoring references for library migration

Automated refactoring is a key feature of modern IDEs. Existing refactorings rely on the transformation of source code declarations, in which references may also be transformed as a side effect. However, there exist situations in which a declaration is not available for refactoring or would be inappropriate to transform, for example, in the presence of dangling references or where a set of references should be retargeted to a different declaration. We investigate the problem of dangling references through a detailed study of three open source libraries. We find that the introduction of dangling references during library migration is a significant real problem, and characterize the specific issues that arise. Based on these findings we provide and test a prototype tool, called Trident, that allows programmers to refactor references. Our results suggest that supporting the direct refactoring of references is a significant improvement over the state-of-the-art.

Domain-Specific Languages for Composable Editor Plugins

Modern IDEs increase developer productivity by incorporating many different kinds of editor services. These can be purely syntactic, such as syntax highlighting, code folding, and an outline for navigation; or they can be based on the language semantics, such as in-line type error reporting and resolving identifier declarations. Building all these services from scratch requires both the extensive knowledge of the sometimes complicated and highly interdependent APIs and extension mechanisms of an IDE framework, and an in-depth understanding of the structure and semantics of the targeted language. This paper describes Spoofax/IMP, a meta-tooling suite that provides high-level domain-specific languages for describing editor services, relieving editor developers from much of the framework-specific programming. Editor services are defined as composable modules of rules coupled to a modular SDF grammar. The composability provided by the SGLR parser and the declaratively defined services allows embedded languages and language extensions to be easily formulated as additional rules extending an existing language definition. The service definitions are used to generate Eclipse editor plugins. We discuss two examples: an editor plugin for WebDSL, a domain-specific language for web applications, and the embedding of WebDSL in Stratego, used for expressing the (static) semantic rules of WebDSL.

Accelerating the creation of customized, language-Specific IDEs in Eclipse

Full-featured integrated development environments have become critical to the adoption of new programming languages. Key to the success of these IDEs is the provision of services tailored to the languages. However, modern IDEs are large and complex, and the cost of constructing one from scratch can be prohibitive. Generators that work from language specifications reduce costs but produce environments that do not fully reflect distinctive language characteristics. We believe that there is a practical middle ground between these extremes that can be effectively addressed by an open, semi-automated strategy to IDE development. This strategy is to reduce the burden of IDE development as much as possible, especially for internal IDE details, while opening opportunities for significant customizations to IDE services. To reduce the effort needed for customization we provide a combination of frameworks, templates, and generators. We demonstrate an extensible IDE architecture that embodies this strategy, and we show that this architecture can be used to produce customized IDEs, with a moderate amount of effort, for a variety of interesting languages.