Industrial Code Bases Research Articles

Nearest‐neighbor, BERT‐based, scalable clone detection: A practical approach for large‐scale industrial code bases

AbstractHidden code clones negatively impact software maintenance, but manually detecting them in large codebases is impractical. Additionally, automated approaches find detection of syntactically‐divergent clones very challenging. While recent deep neural networks (for example BERT‐based artificial neural networks) seem more effective in detecting such clones, their pairwise comparison of every code pair in the target system(s) is inefficient and scales poorly on large codebases. We present SSCD, a BERT‐based clone detection approach that targets high recall of Type 3 and Type 4 clones at a very large scale (in line with our industrial partner's requirements). It computes a representative embedding for each code fragment and finds similar fragments using a nearest neighbor search. Thus, SSCD avoids the pairwise‐comparison bottleneck of other neural network approaches, while also using a parallel, GPU‐accelerated search to tackle scalability. This article describes the approach, proposing and evaluating several refinements to improve Type 3/4 clone detection at scale. It provides a substantial empirical evaluation of the technique, including a speed/efficacy comparison of the approach against SourcererCC and Oreo, the only other neural‐network approach currently capable of scaling to hundreds of millions of LOC. It also includes a large in‐situ evaluation on our industrial collaborator's code base that assesses the original technique, the impact of the proposed refinements and illustrates the impact of incremental, active learning on its efficacy. We find that SSCD is significantly faster and more accurate than SourcererCC and Oreo. SAGA, a GPU‐accelerated traditional clone detection approach, is a little better than SSCD for T1/T2 clones, but substantially worse for T3/T4 clones. Thus, SSCD is both scalable to industrial code sizes, and comparatively more accurate than existing approaches for difficult T3/T4 clone searching. In‐situ evaluation on company datasets shows that SSCD outperforms the baseline approach (CCFinderX) for T3/T4 clones. Whitespace removal and active learning further improve SSCD effectiveness.

Custom static analysis to enhance insight into the usage of in-house libraries

For software maintenance and evolution, insight into the codebase is crucial. One way to enhance insight is the application of static analysis to extract and visualize program-specific relations from the code itself, such as call graphs and inheritance trees. Yet, software often contains in-house libraries: unique, domain-specific libraries whose usage is typically scattered throughout the codebase. To provide sufficient insight into the usage of those libraries, the static analysis must be customized with domain-specific information. In this paper, we propose a method to enhance insight into the usage of in-house libraries by producing custom overviews. Furthermore, we describe three exploratory case studies targeting industrial C++ and Ada codebases, in which the method was developed, evolved, and validated. The method prescribes how to create custom overviews using static analysis iteratively, starting from a user-provided, initial specification of proper library usage using code patterns. As a safeguard, the method includes cross-checks to detect code fragments that deviate from proper library usage. Whenever such a deviating library usage is found, the code owners determine whether that deviating library usage should be added to the specification of proper library usage or the code fragment should be made compliant. The latter alternative makes both the codebase more regular and keeps the custom static analysis simpler. The method creates custom overviews that reveal opportunities to improve the usage of the in-house libraries, e.g., the removal of domain-specific redundant code which cannot be detected using generic tools, such as compilers and linters.We observed that industrial codebases are regular enough to create custom overviews using static analysis in the three exploratory case studies. Furthermore, we observed that the cross-checks, which detect deviating library usage, ensure the validity and completeness of the custom overviews. We conclude that producing custom overviews for in-house libraries using the method is valuable and feasible.

Industrial Experience of Finding Cryptographic Vulnerabilities in Large-scale Codebases

Enterprise environment often screens large-scale (millions of lines of code) codebases with static analysis tools to find bugs and vulnerabilities. Parfait is a static code analysis tool used in Oracle to find security vulnerabilities in industrial codebases. Recently, many studies show that there are complicated cryptographic vulnerabilities caused by misusing cryptographic APIs in Java TM 1 In this paper, we describe how we realize a precise and scalable detection of these complicated cryptographic vulnerabilities based on Parfait framework. The key challenge in the detection of cryptographic vulnerabilities is the high false alarm rate caused by pseudo-influences. Pseudo-influences happen if security-irrelevant constants are used in constructing security-critical values. Static analysis is usually unable to distinguish them from hard-coded constants that expose sensitive information. We tackle this problem by specializing the backward dataflow analysis used in Parfait with refinement insights, an idea from the tool CryptoGuard [ 20 ]. We evaluate our analyzer on a comprehensive Java cryptographic vulnerability benchmark and eleven large real-world applications. The results show that the Parfait-based cryptographic vulnerability detector can find real-world cryptographic vulnerabilities in large-scale codebases with high true-positive rates and low runtime cost.

Recognizing lines of code violating company-specific coding guidelines using machine learning

Software developers in big and medium-size companies are working with millions of lines of code in their codebases. Assuring the quality of this code has shifted from simple defect management to proactive assurance of internal code quality. Although static code analysis and code reviews have been at the forefront of research and practice in this area, code reviews are still an effort-intensive and interpretation-prone activity. The aim of this research is to support code reviews by automatically recognizing company-specific code guidelines violations in large-scale, industrial source code. In our action research project, we constructed a machine-learning-based tool for code analysis where software developers and architects in big and medium-sized companies can use a few examples of source code lines violating code/design guidelines (up to 700 lines of code) to train decision-tree classifiers to find similar violations in their codebases (up to 3 million lines of code). Our action research project consisted of (i) understanding the challenges of two large software development companies, (ii) applying the machine-learning-based tool to detect violations of Sun’s and Google’s coding conventions in the code of three large open source projects implemented in Java, (iii) evaluating the tool on evolving industrial codebase, and (iv) finding the best learning strategies to reduce the cost of training the classifiers. We were able to achieve the average accuracy of over 99% and the average F-score of 0.80 for open source projects when using ca. 40K lines for training the tool. We obtained a similar average F-score of 0.78 for the industrial code but this time using only up to 700 lines of code as a training dataset. Finally, we observed the tool performed visibly better for the rules requiring to understand a single line of code or the context of a few lines (often allowing to reach the F-score of 0.90 or higher). Based on these results, we could observe that this approach can provide modern software development companies with the ability to use examples to teach an algorithm to recognize violations of code/design guidelines and thus increase the number of reviews conducted before the product release. This, in turn, leads to the increased quality of the final software.

A Unified 2D/3D Large-Scale Software Environment for Nonlinear Inverse Problems

Large-scale parameter estimation problems are among some of the most computationally demanding problems in numerical analysis. An academic researcher’s domain-specific knowledge often precludes that of software design, which results in inversion frameworks that are technically correct but not scalable to realistically sized problems. On the other hand, the computational demands for realistic problems result in industrial codebases that are geared solely for high performance, rather than comprehensibility or flexibility. We propose a new software design for inverse problems constrained by partial differential equations that bridges the gap between these two seemingly disparate worlds. A hierarchical and modular design reduces the cognitive burden on the user while exploiting high-performance primitives at the lower levels. Our code has the added benefit of actually reflecting the underlying mathematics of the problem, which lowers the cognitive load on the user using it and reduces the initial startup period before a researcher can be fully productive. We also introduce a new preconditioner for the 3D Helmholtz equation that is suitable for fault-tolerant distributed systems. Numerical experiments on a variety of 2D and 3D test problems demonstrate the effectiveness of this approach on scaling algorithms from small- to large-scale problems with minimal code changes.

Enhancing spark's contract checking facilities using symbolic execution

Spark, a subset of Ada for engineering safety and security-critical systems, is one of the best commercially available frameworks for formal-methods-supported development of critical software. Spark is designed for verification and includes a software contract language for specifying functional properties of procedures. Even though Spark and its static analysis components are beneficial and easy to use, its contract language is rarely used for stating properties beyond simple constraints on scalar values due to the burdens the associated tool support imposes on developers. Symbolic execution (SymExe) techniques have made significant strides in automating reasoning about deep semantic properties of source code. However, most work on SymExe has focused on bug-finding and test case generation as opposed to tasks that are more verification-oriented such as contract checking. In previous work we have presented: (a) SymExe techniques for checking software contracts in embedded critical systems, and (b) Bakar Kiasan, a tool that implements these techniques in an integrated development environment for Spark. In this paper, we give a detailed walk-through of Bakar Kiasan as it is applied to an industrial code base for an embedded security device. We illustrate how Bakar Kiasan provides significant increases in automation, usability, and functionality over existing Spark contract checking tools, and we present results from performance evaluations of its application to industrial examples.