Software Clone Detection Research Articles

Challenging Machine Learning-Based Clone Detectors via Semantic-Preserving Code Transformations

Software clone detection identifies similar code snippets. It has been an active research topic that attracts extensive attention over the last two decades. In recent years, machine learning (ML) based detectors, especially deep learning-based ones, have demonstrated impressive capability on clone detection. It seems that this longstanding problem has already been tamed owing to the advances in ML techniques. In this work, we would like to challenge the robustness of the recent ML-based clone detectors through code semantic-preserving transformations. We first utilize fifteen simple code transformation operators combined with commonly-used heuristics (i.e., Random Search, Genetic Algorithm, and Markov Chain Monte Carlo) to perform equivalent program transformation. Furthermore, we propose a deep reinforcement learning-based sequence generation (DRLSG) strategy to effectively guide the search process of generating clones that could escape from the detection. We then evaluate the ML-based detectors with the pairs of original and generated clones. We realize our method in a framework named CloneGen. CloneGen In evaluation, we challenge the two state-of-the-art ML-based detectors and four traditional detectors with the code clones after semantic-preserving transformations via the aid of CloneGen. Surprisingly, our experiments show that, despite the notable successes achieved by existing clone detectors, the ML models inside these detectors still cannot distinguish numerous clones produced by the code transformations in CloneGen. In addition, adversarial training of ML-based clone detectors using clones generated by CloneGen can improve their robustness and accuracy. CloneGen Meanwhile, compared with the commonly-used heuristics, the DRLSG strategy has shown the best effectiveness in generating code clones to decrease the detection accuracy of the ML-based detectors.

Towards Reliable Code Plagiarism Detection: A Survey on Software Clone Detection

Towards Reliable Code Plagiarism Detection: A Survey on Software Clone Detection

Clone detection through srcClone: A program slicing based approach

Software clone detection is an often used approach to understand and maintain software systems. One category of clones that is challenging to detect but very useful is semantic clones, which are similar in semantics but differ in syntax significantly. Semantic clone detectors have trouble scaling to larger systems and sometimes struggle with recall and precision. To address this, we developed a slice-based scalable approach that detects both syntactic and semantic code clones, srcClone. srcClone ascertains code segment similarity by assessing the similarity of their corresponding program slices. We employ a lightweight, publicly-available, scalable program slicer within our clone detection approach. Using dependency analysis to detect and assess cloned components, we discover insights about code components that can be affected by a clone pair or set. These elements are critical in impact analysis. It can also be used by program analysts to run on non-compilable and incomplete source code, which serves comprehension and maintenance tasks very well. We first evaluate srcClone by comparing it to six state-of-the-art tools and two additional semantic clone detectors in performance, recall, and precision. We use the BigCloneBench real clones benchmark to facilitate comparison. We use an additional 16 established benchmark scenarios to perform a qualitative comparison between srcClone and 44 clone detection approaches in their capabilities to detect these scenarios. To further measure scalability, we evaluate srcClone on 191 versions of Linux kernel, containing approximately 87 MLOC. In our evaluations, we illustrate our approach is both relatively scalable and accurate. While its precision is slightly less than some other techniques, it makes up for it in higher recall including semantic clones unable to be found by any existing techniques. We contend our approach is an important advancement in software cloning that it is both demonstrably scalable and can detect more types of clones than existing work, thus providing developers greater information into their software.

Using Dynamic Time Warping to Detect Clones in Software Systems

Software clone detection is a widely researched area over the last two decades. Code clones are fragments of code judged similar by some metric of similarity. This paper proposes an approach for code clone detection using dynamic time warping technique (i.e., DTW). DTW is a well-known algorithm for aligning and measuring similarity of time series and it has been found effective in many domains where similarity plays an important role such as speech and gesture recognition. The proposed approach finds clones in three steps. First software modules are extracted. Then, the extracted modules are turned to time series. Finally, the time series are compared using the DTW algorithm to find clones. The results of the experiment conducted on a well-known Benchmark show that the approach can detect clones effectively in software systems.

Find Me if You Can: Deep Software Clone Detection by Exploiting the Contest between the Plagiarist and the Detector

Code clone is common in software development, which usually leads to software defects or copyright infringement. Researchers have paid significant attention to code clone detection, and many methods have been proposed. However, the patterns for generating the code clones do not always remain the same. In order to fool the clone detection systems, the plagiarists, known as the clone creator, usually conduct a series of tricky modifications on the code fragments to make the clone difficult to detect. The existing clone detection approaches, which neglects the dynamics of the “contest” between the plagiarist and the detectors, is doomed to be not robust to adversarial revision of the code. In this paper, we propose a novel clone detection approach, namely ACD, to mimic the adversarial process between the plagiarist and the detector, which enables us to not only build strong a clone detector but also model the behavior of the plagiarists. Such a plagiarist model may in turn help to understand the vulnerability of the current software clone detection tools. Experiments show that the learned policy of plagiarist can help us build stronger clone detector, which outperforms the existing clone detection methods.

Code similarity detection through control statement and program features

Software clone detection is an emerging research area in the field of software engineering. Software systems are subjected to continuous modifications in source code to improve the performance of the software, which may lead to code redundancy. Duplicate code/code clone is a piece of code reworked several times in software programs due to copy paste activity or reusability of existing software. Code clone is a prime subject in software evolution. Detection of software clones at the time of software evolution may improve the performance of software and reduce the maintenance cost and effort. This paper proposes metric based methods to detect code clones, as software clone is a universal problem in large scale programming environment. This paper introduces two metric based approaches to detect code clones by comparing (i) Control Statement Features (ii) Program Features like different types of statements, operators and operands. In order to demonstrate the effectiveness of the proposed approaches, extensive experiments are conducted on two datasets, C projects of Bellon's benchmark dataset and student lab programs (SLP).The methods efficiently identify similar functional clones. Proposed models only find similarity of whole programs but intelligent enough to highlight similar code segments across program files.

A Mutation Operator-Based Scenario for Evaluating Software Clone Detection Tools and Techniques

Over the past few years, several software clone detection tools and techniques have been introduced by numerous researchers. The software clone detection techniques and tools are based on their numerous attributes and sub-attributes which make them difficult to complete a comparative study. Therefore, the authors propose a mutation operator-based editing taxonomy for generating different software clone types. In addition, a hypothetical scenario is developed using mutation operator-based editing taxonomy and this hypothetical scenario is used to evaluate various software clone detection techniques and tools. Further, the existing evaluation criterion is extended by the hypothetical scenario which is clearly represented by the analysis of results.

Metaphorical Analysis of Software Clone Detection Techniques based on Dimensions and Metrics

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

Detecting Near Duplicates in Software Documentation

Contemporary software documentation is as complicated as the software itself. During its lifecycle, the documentation accumulates a lot of near duplicate fragments, i.e. chunks of text that were copied from a single source and were later modified in different ways. Such near duplicates decrease documentation quality and thus hamper its further utilization. At the same time, they are hard to detect manually due to their fuzzy nature. In this paper we give a formal definition of near duplicates and present an algorithm for their detection in software documents. This algorithm is based on the exact software clone detection approach: the software clone detection tool Clone Miner was adapted to detect exact duplicates in documents. Then, our algorithm uses these exact duplicates to construct near ones. We evaluate the proposed algorithm using the documentation of 19 open source and commercial projects. Our evaluation is very comprehensive - it covers various documentation types: design and requirement specifications, programming guides and API documentation, user manuals. Overall, the evaluation shows that all kinds of software documentation contain a significant number of both exact and near duplicates. Next, we report on the performed manual analysis of the detected near duplicates for the Linux Kernel Documentation. We present both quantative and qualitative results of this analysis, demonstrate algorithm strengths and weaknesses, and discuss the benefits of duplicate management in software documents.

Обнаружение неточно повторяющегося текста в документации программного обеспечения

Development of software documentation often involves copy-pasting, which produces a lot of duplicate text. Such duplicates make it difficult and expensive documentation maintenance, especially in case of long life cycle of software and its documentation. The situation is further complicated by duplicate information frequently being near duplicate, i.e., the same information may be presented many times with different levels of detail, in various contexts, etc. There are a number approaches to deal with duplicates in software documentation. But most of them use software clone detection technique, that is make difficult to provide efficient near duplicate detection: source code algorithms ignore a document structure, and they produce a lot of false positives. In this paper, we present an algorithm aiming to detect near duplicates in software documentation using natural language processing technique called as N-gramm model. The algorithm has a considerable limitation: it only detects single sentences as near duplicates. But it is very simple and may be easily improved in future. It is implemented with use of Natural Language Toolkit (NLTK), and. Evaluation results are presented for five real life documents from various industrial projects. Manual analysis shows 39 % of false positives in automatic detected duplicates. The algorithm demonstrates reasonable performance: documents of 0,8-3 Mb are processed 5-22 min.

An approach to clone detection in sequence diagrams and its application to security analysis

Duplication in software systems is an important issue in software quality assurance. While many methods for software clone detection in source code and structural models have been described in the literature, little has been done on similarity in the dynamic behaviour of interactive systems. In this paper, we present an approach to identifying near-miss interaction clones in reverse-engineered UML sequence diagrams. Our goal is to identify patterns of interaction (“conversations”) that can be used to characterize and abstract the run-time behaviour of web applications and other interactive systems. In order to leverage existing robust near-miss code clone technology, our approach is text-based, working on the level of XMI, the standard interchange serialization for UML. Clone detection in UML behavioural models, such as sequence diagrams, presents a number of challenges—first, it is not clear how to break a continuous stream of interaction between lifelines (representing the objects or actors in the system) into meaningful conversational units. Second, unlike programming languages, the XMI text representation for UML is highly non-local, using attributes to reference-related elements in the model file remotely. In this work, we use a set of contextualizing source transformations on the XMI text representation to localize related elements, exposing the hidden hierarchical structure of the model and allowing us to granularize behavioural interactions into conversational units. Then we adapt NICAD, a robust near-miss code clone detection tool, to help us identify conversational clones in reverse-engineered behavioural models. These conversational clones are then analysed to find worrisome interactions that may indicate security access violations.

On fuzzy repetitions detection in documentation reuse

Increasing complexity of software documentation calls for additional requirements of document maintenance. Documentation reuse can make a considerable contribution to solve this problem. This paper presents a method for fuzzy repetitions search in software documentation that is based on software clone detection. The search results are used for document refactoring. This paper also presents Documentation Refactoring Toolkit implementing the proposed method and integrated with the DocLine project. The proposed approach is evaluated on documentation packages for a number of open-source projects: Linux Kernel, Zend Framework, Subversion, and DocBook.

Hybrid approach for Detecting Code Clone by Metric and Token based comparison

In software development process, coping of existing code fragment and pasting them with or without modification is a frequent process. Clone means copy of an original form or duplicate. Software clone detection is important to reduce the software maintenance cost and to recognize the software system in a better way. There are many software code clone detection techniques such as textbased, token-based, Abstract Syntax tree based etc. and they are used to spot and finding the existence of clones in software system. One of the approaches to detect code clones is by analysis of different metrics for the programs. Another approach is token based comparisons of two programs to detect code clone. Our technique uses the combination of metrics and token based approach using hashing algorithm by analysis of two source codes in the process of finding clones among them and the percentage of cloning is reported as result. Keywords— software clone detection, metric based comparison, token based approach using hashing algorithm, java byte code.

Code Clone Detection using Sequential Pattern Mining

ABSTRACT This paper presents a new technique for clone detection using sequential pattern mining titled EgyCD. Over the last decade many techniques and tools for software clone detection have been proposed such as textual approaches, lexical approaches, syntactic approaches, semantic approaches …, etc. In this paper, we explore the potential of data mining techniques in clone detection. In particular, we developed a clone detection technique based on sequential pattern mining (SPM). The source code is treated as a sequence of transactions processed by the SPM algorithm to find frequent itemsets. We run three experiments to discover code clones of Type I, Type II and Type III and for plagiarism detection. We compared the results with other established code clone detectors. Our technique discovers all code clones in the source code and hence it is slower than the compared code clone detectors since they discover few code clones compared with EgyCD. Keywords

An Efficient Software Clone Detection System based on the Textual Comparison of Dynamic Methods and Metrics Computation

clone is a kind of software recycling that has a greater influence on the maintenance of huge software systems. The business services are provided by web applications that employ a combination of page design and language code scripting. Usually, code redundancy in applications result from copy and paste practices called code clones. Searching for software clones is the key objective of this research. The clone identification procedure which is introduced in this paper is a hybrid approach that depends on template conversion and metrics comparison. There are four phases involved in the proposed scheme, namely, input and pre-processing, template conversion, metrics computation and clone type detection. File integration, elimination of white noise and statement normalization are the steps involved in the pre-processing stage. The metrics that are calculated includes numerous lines of code, arguments, looping statements, return statements and conditional statements. The pairs that have identical characteristics during textual evaluation are termed as the clones. This method of clone detection seems to be less complex with better accuracy and efficiency in contrast to other existing methods. The performance analysis is made against the prevailing systems to show efficiency improvement obtained through this method. The implementations are carried out with the help of JAVA. Keywordsdetection, Template conversion, Metrics computation, File integration, Normalization.

A Comparative Analysis of Software Clone Management Techniques

Software clone research has proved that there are redundancies in software. This redundancy increased the maintenance effort. For nearly past two decades a number of software clone detection techniques and clone management techniques have been proposed in the software clone research. In literature some researchers has also proposed clone management techniques such as clone removal, clone modification, analyses the effect of clones during maintainability, investigating their evolution, and assessing the root causes of clones. There have been a number of researches which also focused on the evaluation of clone detection approaches. This paper is the analysis of different clone detection and management techniques. First, we analyze and evaluate all the currently available clone detection techniques and tools. Second, we study and discuss the different clone management techniques and tools which are currently available.

Software Clone Detection and Refactoring

Several studies have been proposed in the literature on software clones from different points of view and covering many correlated features and areas, which are particularly relevant to software maintenance and evolution. In this paper, we describe our experience on clone detection through three different tools and investigate the impact of clone refactoring on different software quality metrics.

SOFTWARE CODE CLONE DETECTION MODEL USING HYBRID APPROACH

The aspiration of this study is to understand and analyze the concept of software Cloning and its detection. Software cloning is an acuity in which source code is duplicated. Software cloning and its detection is one of emerging and most dominant area of research in the field of software engineering. There exist numbers of techniques to detect clone in software. The focus of this study will be given on acquiring and analyzing the concept of hybrid clone detection technique. We will try to devise an algorithm for detecting duplicacy in the software by using hybrid software clone detection technique. The algorithm will first compute the required software metrics that provide sufficient information regarding the software application and then depending on software metrics matches the clone will be detected. While detecting clone we will focus on line clone rather than token or word.

HYBRID TECHNIQUE FOR SOFTWARE CODE CLONE DETECTION

Software Clone detection is one of the hottest research area that helps in detecting duplicate code from an applications. The research has shown that 5% to 20% of software systems can contain duplicated code that is generated by simply copying the existing program code and pasting with or without minor modifications. Cloning creates problem when a bug is found in one code segment that was copied and pasted at several locations earlier. The objective of this study is to analyze the working of hybrid clone detection technique that design and analyze a hybrid technique for detecting software clone in an application. We will combine metric approach with text base (line of code) technique for the above said reason. A model will be designed to automate the concept of clone detection.

Software Bertillonage

Deployed software systems are typically composed of many pieces, not all of which may have been created by the main development team. Often, the provenance of included components--such as external libraries or cloned source code--is not clearly stated, and this uncertainty can introduce technical and ethical concerns that make it difficult for system owners and other stakeholders to manage their software assets. In this work, we motivate the need for the recovery of the provenance of software entities by a broad set of techniques that could include signature matching, source code fact extraction, software clone detection, call flow graph matching, string matching, historical analyses, and other techniques. We liken our provenance goals to that of Bertillonage, a simple and approximate forensic analysis technique based on bio-metrics that was developed in 19th century France before the advent of fingerprints. As an example, we have developed a fast, simple, and approximate technique called anchored signature matching for identifying the source origin of binary libraries within a given Java application. This technique involves a type of structured signature matching performed against a database of candidates drawn from the Maven2 repository, a 275 GB collection of open source Java libraries. To show the approach is both valid and effective, we conducted an empirical study on 945 jars from the Debian GNU/Linux distribution, as well as an industrial case study on 81 jars from an e-commerce application.