Source Code Fragments Research Articles

Identifying Concurrent Behaviours in Erlang Legacy Systems

In Erlang, behaviours are special forms of design patterns. There are many benefits to using behaviours. For example, behaviours can help abstract away the most common parts when solving similar problems. Design pattern recognition may help understand the source code of the software. It can provide structured information about the purpose of specific parts and the design decisions behind the implementation. For object-oriented languages, several tools exist that use different approaches and methods to identify design patterns. We present a method for identifying source code fragments in legacy Erlang systems amenable to transforming into client-server Erlang design patterns. In our analysis, we identify the base set of server candidates using concurrent process analysis and narrow down the result using further static analysis knowledge using the RefactorErl framework.

Enhancing vulnerability detection via AST decomposition and neural sub-tree encoding

The explosive growth of software vulnerabilities poses a serious threat to the system security and has become one of the urgent problems of the day. However, existing vulnerability detection methods are still faced with limitations in reaching the balance between detection accuracy, efficiency and applicability. Following a divide-and-conquer strategy, this paper proposes TrVD (abstract syntax Tree decomposition based Vulnerability Detector) to disclose the indicative semantics implied in the source code fragments for accurate and efficient vulnerability detection. To facilitate the capture of subtle semantic features, TrVD converts the AST of a code fragment into an ordered set of sub-trees of restricted sizes and depths with a novel decomposition algorithm. The semantics of each sub-tree can thus be effectively collected with a carefully designed tree-structured neural network. Finally, a Transformer-style encoder is utilized to aggregate the long-range contextual semantics of all sub-trees into a vulnerability-specific vector to represent the target code fragment. The extensive experiments conducted on five large datasets consisting of diverse real-world and synthetic vulnerable samples demonstrate the performance superiority of TrVD against SOTA approaches in detecting the presence of vulnerabilities and pinpointing the vulnerability types. The ablation studies also confirm the effectiveness of TrVD’s core designs.

ASTENS-BWA: Searching partial syntactic similar regions between source code fragments via AST-based encoded sequence alignment

Code search is a common activity in software development, and code-to-code search can benefit in a wide range of use-case scenarios. Code-to-code search uses a code fragment as the query for searching similar code fragments from large corpora. The results of a search can be applied to some software engineering tasks, such as search-based code recommendation, data-driven program repairing, and software plagiarism detection. To be put into daily use, the code-to-code search needs to find similar code fragments accurately and efficiently in a large dataset. Some search engines can locate exactly similar code, but are not able to search syntactical clones. Therefore, we propose ASTENS-BWA, a novel approach for searching syntactic similar code regions between code fragments via a tree-based sequence alignment. Source code has been transformed into a tree-based sequence that contains the structure information, and a sequence alignment algorithm has been applied to find similar regions. We evaluate ASTENS-BWA on three different tasks, the results demonstrate that our approach can find syntactical similar regions for programming code and retrieve similar code fragments fast and with high accuracy. As a code clone detection tool, ASTENS-BWA can report clone pairs in a high recall, but it needs manually check to reduce the false alarms. ASTENS-BWA is scalable and can report cloned code fragments in seconds for a code corpus of million lines of code. • Novel code clone search approach that scales to large source code repositories. • Quick searching syntactical clones between code fragments via tree-based representation. • Code clone search approach that can outperform state-of-the-art techniques. • Application of the proposed approach towards code recommendation and clone detection.

Improving Large-Gap Clone Detection Recall Using Multiple Features

Code clone refers to two or more identical or similar source code fragments. Research on code clone detection has lasted for decades. Investigation and evaluation of existing clone detection techniques indicate that they are resilient to function-level clone detection. Still, there may be room for further research in block-level clone detection. Particularly, type-3 clones that include large gaps, are ongoing challenges. To solve these problems, we propose a clone detection method based on multiple code features. It aims to improve the recall rate of code block clone detection and overcome large-gap and hard-to-detect type-3 clones. This method first splits the source code files based on the program’s structural features and context features to obtain code blocks. The collection of code blocks obtained in this way is complete, and the large gaps in clone pairs will also be removed. In addition, we only need to compute the similarity between code blocks with the same structural features, which can also significantly save time and resources. The similarity is obtained by calculating the proportion of the same tokens between two code blocks. Moreover, since different types of tokens have different weights in similarity calculation, we use supervised learning to obtain a classifier model between token features and code clone. We divide the tokens into 13 types and train the machine learning model with the manually confirmed clone or non-clone pair. Finally, we develop a prototype system and compare our tools with existing tools under the Mutation Framework and in several actual C projects. The experimental results also demonstrate the advancement and practicality of our prototype.

DETECTION OF SOURCE CODE IN INTERNET TEXTS USING AUTOMATICALLY GENERATED MACHINE LEARNING MODELS

In the paper, the authors are presenting the outcome of web scraping software allowing for the automated classification of source code. The software system was prepared for a discussion forum for software developers to find fragments of source code that were published without marking them as code snippets. The analyzer software is using a Machine Learning binary classification model for differentiating between a programming language source code and highly technical text about software. The analyzer model was prepared using the AutoML subsystem without human intervention and fine-tuning and its accuracy in a described problem exceeds 95%. The analyzer based on the automatically generated model has been deployed and after the first year of continuous operation, its False Positive Rate is less than 3%. The similar process may be introduced in document management in software development process, where automatic tagging and search for code or pseudo-code may be useful for archiving purposes.

Towards Improving Code Stylometry Analysis in Underground Forums

Abstract Code Stylometry has emerged as a powerful mechanism to identify programmers. While there have been significant advances in the field, existing mechanisms underperform in challenging domains. One such domain is studying the provenance of code shared in underground forums, where code posts tend to have small or incomplete source code fragments. This paper proposes a method designed to deal with the idiosyncrasies of code snippets shared in these forums. Our system fuses a forum-specific learning pipeline with Conformal Prediction to generate predictions with precise confidence levels as a novelty. We see that identifying unreliable code snippets is paramount to generate high-accuracy predictions, and this is a task where traditional learning settings fail. Overall, our method performs as twice as well as the state-of-the-art in a constrained setting with a large number of authors (i.e., 100). When dealing with a smaller number of authors (i.e., 20), it performs at high accuracy (89%). We also evaluate our work on an open-world assumption and see that our method is more effective at retaining samples.

A Simulation Model for Macro- and Micro-Fusion Algorithms in the CPU Core

The article discusses the features of modern processor’s microarchitecture, the method of instruction’s and micro-operation’s accelerated execution. The research focuses on the organization of the decoding stage in the CPU core pipeline and Macro- and Micro-fusion algorithms. The Macro- and Micro-fusion mechanisms are defined. A computer simulator has been developed to explore these mechanisms. The developed software has a user-friendly interface, is easy to use, and combines training and research options. The computer simulator demonstrates the sequence of mechanism’ s implementation; the resulting macro-or microoperations set after Macro- and Micro-fusion, and also reflects each algorithm features for different processor’s families. The software allows you to use either a pre-prepared file with Assembler (x86) code fragments as source data, or enter/change the source code fragments at your request. The main combinations of machine instructions that can be fused into a single macro-operation are considered, as well as instructions that can be decoded into fused micro-operations. The simulator can be useful both for in Computer Science & Engineering students, especially for on-line education and for researchers and General-purpose CPU cores developers.

Flowchart-Based Cross-Language Source Code Similarity Detection

Source code similarity detection has various applications in code plagiarism detection and software intellectual property protection. In computer programming teaching, students may convert the source code written in one programming language into another language for their code assignment submission. Existing similarity measures of source code written in the same language are not applicable for the cross-language code similarity detection because of syntactic differences among different programming languages. Meanwhile, existing cross-language source similarity detection approaches are susceptible to complex code obfuscation techniques, such as replacing equivalent control structure and adding redundant statements. To solve this problem, we propose a cross-language code similarity detection (CLCSD) approach based on code flowcharts. In general, two source code fragments written in different programming languages are transformed into standardized code flowcharts (SCFC), and their similarity is obtained by measuring their corresponding SCFC. More specifically, we first introduce the standardized code flowchart (SCFC) model to be the uniform flowcharts representation of source code written in different languages. SCFC is language-independent, and therefore, it can be used as the intermediate structure for source code similarity detection. Meanwhile, transformation techniques are given to transform source code written in a specific programming language into an SCFC. Second, we propose the SCFC-SPGK algorithm based on the shortest path graph kernel to measure the similarity between two SCFCs. Thus, the similarity between two pieces of source code in different programming languages is given by the similarity between SCFCs. Experimental results show that compared with existing approaches, CLCSD has higher accuracy in cross-language source code similarity detection. Furthermore, CLCSD cannot only handle common source code obfuscation techniques used by students in computer programming teaching but also obtain nearly 90% accuracy in dealing with some complex obfuscation techniques.

On the diversity and frequency of code related to mathematical formulas in real-world Java projects

In this paper, the term formula code refers to fragments of source code that implement a mathematical formula. We present empirical studies that analyze the diversity and frequency of formula code in open-source-software projects. In an exploratory study, we investigated what kinds of formulas are implemented in real-world Java projects and derived syntactical patterns and constraints. We refined these patterns for sum and product formulas to automatically detect formula code in software archives and to reconstruct the implemented formula in mathematical notation. In a quantitative study of a large sample of engineered Java projects on GitHub we analyzed the frequency of formula code and estimated that one of 700 lines of code in this sample implements a sum or product formula. For a sample of scientific-computing projects, we found that one of 100 lines of code implements a sum or product formula. To assess the need for tool support, we investigated the helpfulness of comments for program understanding in a sample of formula-code fragments and performed an online survey. Our findings provide first insights into the characteristics of formula code, that can motivate further studies on the role of formula code in software projects and the design of formula-related tools.

A universal cross language software similarity detector for open source software categorization

While there are novel approaches for detecting and categorizing similar software applications, previous research focused on detecting similarity in applications written in the same programming language and not on detecting similarity in applications written in different programming languages. Cross-language software similarity detection is inherently more challenging due to variations in language, application structures, support libraries used, and naming conventions. In this paper we propose a novel model, CroLSim, to detect similar software applications across different programming languages. We define a semantic relationship among cross-language libraries and API methods (both local and third party) using functional descriptions and a word-vector learning model. Our experiments show that CroLSim can successfully detect cross-language similar software applications, which outperforms all existing approaches (mean average precision rate of 0.65, confidence rate of 3.6, and 75% highly rated successful queries). Furthermore, we applied CroLSim to a source code repository to see whether our model can recommend cross-language source code fragments if queried directly with source code. From our experiments we found that CroLSim can recommend cross-language functional similar source code when source code is directly used as a query (average precision=0.28, recall=0.85, and F-Measure=0.40).

SPELLing out energy leaks: Aiding developers locate energy inefficient code

Although hardware is generally seen as the main culprit for a computer’s energy usage, software too has a tremendous impact on the energy spent. Unfortunately, there is still not enough support for software developers so they can make their code more energy-aware.This paper proposes a technique to detect energy inefficient fragments in the source code of a software system. Test cases are executed to obtain energy consumption measurements, and a statistical method, based on spectrum-based fault localization, is introduced to relate energy consumption to the source code. The result of our technique is an energy ranking of source code fragments pointing developers to possible energy leaks in their code. This technique was implemented in the SPELL toolkit.Finally, in order to evaluate our technique, we conducted an empirical study where we asked participants to optimize the energy efficiency of a software system using our tool, while also having two other groups using no tool assistance and a profiler, respectively. We showed statistical evidence that developers using our technique were able to improve the energy efficiency by 43% on average, and even out performing a profiler for energy optimization.

Enhancing code clone detection using control flow graphs

Code clones are syntactically or semantically equivalent code fragments of source code. Copy-and-paste programming allows software developers to improve development productivity, but it could produce code clones that can introduce non-trivial difficulties in software maintenance. In this paper, a code clone detection framework is presented with a feature extractor and a clone classifier using deep learning. The clone classifier is trained with true and false clones and then is tested with a test dataset to evaluate the performance of the proposed approach to clone detection. In particular, the proposed approach to clone detection uses Control Flow Graphs (CFGs) to extract features of a given code snippet. The selected features are used to compute similarity scores for comparing two code fragments. The clone classifier is trained and tested with similarity scores that quantify the degree of how similar two code fragments are. The experimental results demonstrate that using CFG features is a viable methodology in terms of the effectiveness of clone detection for both syntactic and semantic clones.

Git Blame Who?: Stylistic Authorship Attribution of Small, Incomplete Source Code Fragments

Abstract Program authorship attribution has implications for the privacy of programmers who wish to contribute code anonymously. While previous work has shown that individually authored complete files can be attributed, these efforts have focused on such ideal data sets as contest submissions and student assignments. We explore the problem of authorship attribution “in the wild,” examining source code obtained from open-source version control systems, and investigate how contributions can be attributed to their authors, either on an individual or a per-account basis. In this work, we present a study of attribution of code collected from collaborative environments and identify factors which make attribution of code fragments more or less successful. For individual contributions, we show that previous methods (adapted to be applied to short code fragments) yield an accuracy of approximately 50% or 60%, depending on whether we average by sample or by author, at identifying the correct author out of a set of 104 programmers. By ensembling the classification probabilities of a sufficiently large set of samples belonging to the same author we achieve much higher accuracy for assigning the set of samples to the correct author from a known suspect set. Additionally, we propose the use of calibration curves to identify which samples are by unknown and previously unencountered authors.

Using Stack Overflow content to assist in code review

SummaryAn essential goal for programmers is to minimize the cost of identifying and correcting defects in source code. Code review is commonly used for identifying programming defects. However, manual code review has some shortcomings: (1) it is time‐consuming and (2) outcomes are subjective and depend on the skills of reviewers. An automated approach for assisting in code reviews is thus highly desirable. We present a tool for assisting in code review and results from our experiments evaluating the tool in different scenarios. The tool leveraged content available from professional programmer support forums (eg, StackOverflow.com) to determine potential defectiveness of a given piece of source code. The defectiveness is expressed on the scale of {Likely defective, neutral, unlikely to be defective}. The basic idea employed in the tool is (1) to identify a set P of discussion posts on Stack Overflow such that each p∈P contains source code fragment(s), which sufficiently resemble the input code C being reviewed, and (2) to determine the likelihood of C being defective by considering all p∈P. A novel aspect of our approach is to use document fingerprinting for comparing two pieces of source code. Our choice of document fingerprinting technique is inspired by source code plagiarism detection tools where it has proven to be very successful. In the experiments that we performed to verify the effectiveness of our approach, source code samples from more than 300 GitHub open‐source repositories were taken as input. An F1 score of 0.94 has been achieved in identifying correct/relevant results.

Structured information in bug report descriptions—influence on IR-based bug localization and developers

Multiple information retrieval (IR)-based bug localization techniques have been proposed over the last years. The foundation of the approaches relies on textual similarity of the bug report description and the source code files. The basic assumption is that these descriptions are well suited to query the code base. However, often bug reports contain structured information such as stack traces and source code next to natural language, which might interfere with the initial belief. In this paper, we systematically analyze the influence of structured information on IR-based techniques. Therefore, an empirical study on 7334 bug reports, out of which more than 30% contain structured information, was carried out. Based on the results, a follow-up user study was conducted focusing on source code fragments found in bug reports. Our results show that stack traces tend to negatively affect IR-based bug localization performance and require special handling. Compared to natural language–only reports, source code is beneficial for IR-based algorithms, as well as for developers to identify false positives in bug localization results.

Plagiarism detection in students’ programming assignments based on semantics: multimedia e-learning based smart assessment methodology

The multimedia-based e-Learning methodology provides virtual classrooms to students. The teacher uploads learning materials, programming assignments and quizzes on university’ Learning Management System (LMS). The students learn lessons from uploaded videos and then solve the given programming tasks and quizzes. The source code plagiarism is a serious threat to academia. However, identifying similar source code fragments between different programming languages is a challenging task. To solve the problem, this paper proposed a new plagiarism detection technique between C++ and Java source codes based on semantics in multimedia-based e-Learning and smart assessment methodology. First, it transforms source codes into tokens to calculate semantic similarity in token by token comparison. After that, it finds semantic similarity in scalar value for the complete source codes written in C++ and Java. To analyse the experiment, we have taken the dataset consists of four (4) case studies of Factorial, Bubble Sort, Binary Search and Stack data structure in both C++ and Java. The entire experiment is done in R Studio with R version 3.4.2. The experimental results show better semantic similarity results for plagiarism detection based on comparison.

Identification of Potential Reusable Subroutines in Recorded Automated Test Scripts

In the automated testing based on actions in user interface of the tested application, one of the key challenges is maintenance of these tests. The maintenance overhead can be decreased by suitably structuring the test scripts, typically by employing reusable objects. To aid in the development, maintenance and refactoring of these test scripts, potentially reusable objects can be identified by a semi-automated process. In this paper, we propose a solution that identifies the potentially reusable objects in a set of automated test scripts and then provides developers with suggestions about these objects. During this process, we analyze the semantics of specific test steps using a system of abstract signatures. The solution can be used to identify the potentially reusable objects in both recorded automated test sets and tests programmed in an unstructured style. Moreover, compared to approaches that are based solely on searching for repetitive source code fragments, the proposed system identifies potentially reusable objects that are more relevant for test automation.

An Approach to Source Code Plagiarism Detection Based on Abstract Implementation Structure Diagram

Source-code plagiarism detection in programming, concerns the identification of source-code files that contain similar and/or identical source-code fragments. Based on the analysis of the characteristics and defects of the existing program code similarity detection system, a method of source code similarity detection based on Abstract Implementation Structure Diagram (AISD) is proposed. The source code modelling and format into an abstract implementation structure diagram, and forming structural feature strings and variable reference relationship sequences by extracting structural features and variable position features. We calculate the overall similarity by calculating structural similarity and variable similarity. The results demonstrate that the performance of the proposed AISD-based approach overcomes other approaches on the same source code datasets, and reveals promising results as an efficient and reliable approach to source-code plagiarism detection.

Just enough semantics: An information theoretic approach for IR-based software bug localization

ContextSoftware systems are often shipped with defects. Whenever a bug is reported, developers use the information available in the associated report to locate source code fragments that need to be modified in order to fix the bug. However, as software systems evolve in size and complexity, bug localization can become a tedious and time-consuming process. To minimize the manual effort, contemporary bug localization tools utilize Information Retrieval (IR) methods for automated support. IR methods exploit the textual content of bug reports to automatically capture and rank relevant buggy source files. ObjectiveIn this paper, we propose a new paradigm of information-theoretic IR methods to support bug localization tasks in software systems. These methods, including Pointwise Mutual Information (PMI) and Normalized Google Distance (NGD), exploit the co-occurrence patterns of code terms in the software system to reveal hidden textual semantic dimensions that other methods often fail to capture. Our objective is establish accurate semantic similarity relations between source code and bug reports. MethodFive benchmark datasets from different application domains are used to conduct our analysis. The proposed methods are compared against classical IR methods that are commonly used in bug localization research. ResultsThe results show that information-theoretic IR methods significantly outperform classical IR methods, providing a semantically aware, yet, computationally efficient solution for bug localization in large and complex software systems. (A replication package is available at: http://seel.cse.lsu.edu/data/ist17.zip). ConclusionsInformation-theoretic co-occurrence methods provide “just enough semantics” necessary to establish relations between bug reports and code artifacts, achieving a balance between simple lexical methods and computationally-expensive semantic IR methods that require substantial amounts of data to function properly.

Design Pattern Instances within Model Driven Development Based on Abstraction, Concretization and Variability

The main goal of the paper is to present the method of design pattern support based on principles of model driven development: the abstraction, semantics and model transformations. More specifically, the method is based on the principle of suggestion of design pattern instances via the semantic marking of model elements or source code fragments and on the subsequent transformations of this way marked models or source code. Thanks to the continual support of the design patterns at more levels of abstraction and thanks to the transformations between particular model levels and source code, the method is targeted to achieve the applicability in the area of the iterative, incremental and model driven development.