Bytecode Level Research Articles

TxMirror: When the Dynamic EVM Stack Meets Transactions for Smart Contract Vulnerability Detection

Smart contract vulnerability can be exploited to cause serious financial loss. While there are many logic-based techniques for detecting vulnerabilities, few have focused on the dynamic stack of the Ethereum virtual machine (EVM) in the process of transactions. This motivates us to raise an intriguing question: What will happen when the dynamic EVM stack meets a transaction for smart contract vulnerability detection? To answer it, we propose a novel data-driven framework, dubbed TxMirror, to detect smart contract vulnerability at the bytecode level by simulating transactions symmetrically. Beyond logic-based wisdom, TxMirror customizes EVM for smart contract vulnerability detection, and stores its dependency between the stack data and logic relation in a new manner, that is, all the data are stored in a double link forest and can index the logic that creates them; it directly inspects customized EVM stack data in a transaction without replaying history transactions repeatedly or recording EVM bytecode-level traces. Furthermore, it extends detection rules defined by user interests, possessing good adaptability for developers. Extensive experiments demonstrate that TxMirror effectively detects attacks and vulnerabilities in unpredictable smart contracts.

Effective Isolation of Fault-Correlated Variables via Statistical and Mutation Analysis

It is a widely-adopted strategy for developers to monitor the values of program variables when debugging in practice. In particular, developers often set breakpoints at specific locations or execute the program step by step in the debugging mode to inspect if abnormal values or status will be observed for concerned variables. Such a practical debugging strategy can facilitate developers in understanding and localizing the target fault. This study aims to identify suspicious program variables of a given fault (i.e., denoted as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fault-correlated variables</i> ) automatically, thus facilitating the debugging activities for developers. To the best of our knowledge, this is the finest granularity in fault localization (FL) so far, which can address the limitations of being coarse-grained as faced by existing FL techniques. However, isolating fault-correlated variables precisely is challenging since there are usually substantially different variables used or defined in a program, and plenty of them are in the same basic block which cannot be well discriminated from each other since they will be either executed or not against the given test suite. To address such challenges, this study presents IsoVar, a two-phase model to isolate fault-correlated variables. Specifically, IsoVar first performs statistical analysis based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">variable execution matrices</i> , which is a novel concept proposed in this study, to identify a set of suspicious variables. It then observes the impacts of those variables on the program dynamically after applying subtle mutations at the bytecode level, to further isolate fault-correlated variables. Extensive experiments on Defects4J and Bears demonstrate that IsoVar can outperform state-of-the-art techniques significantly ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$13.0\%$</tex-math></inline-formula> for MAP and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$19.3\%$</tex-math></inline-formula> for MRR). More importantly, we incorporated IsoVar into 11 existing FL techniques as well as 14 automated program repair techniques, and found that IsoVar can significantly boost their performance.

Android Malware Detection in Bytecode Level Using TF-IDF and XGBoost

Abstract Android is the dominant operating system in the smartphone market and there exists millions of applications in various application stores. The increase in the number of applications has necessitated the detection of malicious applications in a short time. As opposed to dynamic analysis, it is possible to obtain results in a shorter time in static analysis as there is no need to run the applications. However, obtaining various information from application packages using reverse engineering techniques still requires a substantial amount of processing power. Although some attempts have been made to solve this problem by analyzing binary files without decoding the source code, there is still more work to be done in this area. In this study, we analyzed the applications in bytecode level without decoding the binary source files. We proposed a model using Term Frequency - Inverse Document Frequency (TF-IDF) word representation for feature extraction and Extreme Gradient Boosting (XGBoost) method for classification. The experimental results show that our model classifies a given application package as a malware or benign in 2.75 s with 99.05% F1-score on a balanced dataset, and in 3.30 s with 99.35% F1-score on an imbalanced dataset containing obfuscated malwares.

Don’t run on fumes—Parametric gas bounds for smart contracts

Gas is a measurement unit of the computational effort that it will take to execute every single replicated operation that takes part in the Ethereum blockchain platform. If a transaction exceeds the amount of gas allotted by the user (known as gas limit), an out-of-gas exception is raised and its execution is interrupted. One of the main open problems in the analysis of Ethereum smart contracts is the inference of sound bounds on their gas consumption.We present, to the best of our knowledge, the first static analysis that is able to infer sound parametric (i.e., non-constant) gas bounds for smart contracts. The inferred bounds can be parametric on the sizes of the input parameters for the functions, but also they can be parametric on the contract state, or blockchain data. Our gas analysis is developed at EVM bytecode level, in which Ethereum gas model is defined.Our analysis is implemented in a tool named Gastap, Gas-Aware Smart contracT Analysis Platform, which takes as input a smart contract and automatically infers sound gas upper-bounds for its public functions. Gastap has been applied over 318,093 functions fetched from the Ethereum blockchain, and succeeded to obtain gas bounds for 90.24% of them.

Scalable online vetting of Android apps for measuring declared SDK versions and their consistency with API calls

Android has been the most popular smartphone system with multiple platform versions active in the market. To manage the application’s compatibility with one or more platform versions, Android allows apps to declare the supported platform SDK versions in their manifest files. In this paper, we conduct a systematic study of this modern software mechanism. Our objective is to measure the current practice of declared SDK versions (which we term as DSDK versions afterwards) in real apps, and the (in)consistency between DSDK versions and their host apps’ API calls. To successfully analyze a modern dataset of 22,687 popular apps (with an average app size of 25MB), we design a scalable approach that operates on the Android bytecode level and employs a lightweight bytecode search for app analysis. This approach achieves a good performance suitable for online vetting in app markets, requiring only around 5 seconds to process an app on average. Besides shedding light on the characteristics of DSDK in the wild, our study quantitatively measures two side effects of inappropriate DSDK versions: (i) around 35% apps under-set the minimum DSDK versions and could incur runtime crashes, but fortunately, only 11.3% apps could crash on Android 6.0 and above; (ii) around 2% apps, due to under-claiming the targeted DSDK versions, are potentially exploitable by remote code execution, and half of them invoke the vulnerable API via embedded third-party libraries. These results indicate the importance and difficulty of declaring correct DSDK, and our work can help developers fulfill this goal.

Compile-time code virtualization for android applications

Infringing intellectual property by reverse analysis is a severe threat to Android applications. By replacing the program instructions with virtual instructions that an adversary is unfamiliar with, code obfuscation based on virtualization is a promising way of protecting Android applications against reverse engineering. However, the current code virtualization approaches for Android only target at the DEX bytecode level. The DEX file with the open file format and more semantic information makes the decode-dispatch pattern easier to expose, which has been identified as a severe vulnerability of security and can be exploited by various attacks. Further, decode-dispatch interpretation frequently uses indirect branches in this structure to introduce extra overhead. This paper presents a novel approach to transfer code virtualization from DEX level to native level, which possesses strong security strength and good stealth, with only modest cost. Our approach contains two components: pre-compilation and compile-time virtualization. Pre-compilation is designed for performance improvement by identifying and decompiling the critical functions which consume a significant fraction of execution time. Compile-time virtualization builds upon the widely used LLVM compiler framework. It automatically translates the DEX bytecode into the common LLVM intermediate representations where a unified code virtualization pass can be applied for DEX code. We have implemented a working prototype Dex2VM of our technique and applied it to eight representative Android applications. Our experimental results show that the proposed approach can effectively protect the target code against a state-of-the-art code reverse engineering tool that is specifically designed for code virtualization, and it achieves good stealth with only modest cost.

Refactoring Java Programs for Customizable Locks Based on Bytecode Transformation

Developers of parallel programming are faced with choices of using various Java locks. Choosing the best lock is a challenging task because a multithreaded application implemented using different locks may end up with uncertain performance. There is a strong need for automated tool support that helps Java programmers choose the best lock for each specific application. This paper presents an automated transformation approach to convert an implementation using a synchronized lock to a ReentrantLock or a ReentrantReadWriteLock at the bytecode level. For the ReentrantLock, our approach runs analysis for global monitors using the visitor pattern in the Joeq compiler. For the ReentrantReadWriteLock, a read or write lock is chosen after a side-effect analysis. Then, the proposed system validates the consistency of the analysis results to ensure a correct sequence of lock usage. Finally, lock operations and related thread communication operations are transformed into a ReentrantLock or a ReentrantReadWriteLock using a bytecode transformation framework Javassist. We validate our approach on several benchmarks including RBTree, PC, SPECjbb2005, and HSQLDB. The experimental results show that the proposed automated transformation approach can transform these benchmarks successfully in a timely fashion.

Exploiting Binary-Level Code Virtualization to Protect Android Applications Against App Repackaging

Application repackaging is a severe problem for Android systems. Many Android malware programs pass the mobile platform fundamental security barriers through repackaging other legitimate apps. Most of the existing anti-repackaging schemes only work at the Android DEX bytecode level, but not for the shared object files consisting of native ARM-based machine instructions. Lacking the protection at the native machine code level opens a door for attackers to launch repackaging attacks on the shared libraries that are commonly used on Android apps. This paper presents CodeCloak, a novel anti-repackaging system to protect Android apps at the native code level. CodeCloak employs binary-level code virtualization techniques to protect the target application. At the native machine code level, it uses a newly designed stack-based virtualization structure to obfuscate and protect critical algorithm implementations that have been compiled into native instructions. It leverages multiple dynamic code protection schemes to increase the diversity of the program behavior at runtime, aiming to increase the difficulties for performing code reverse engineering. We evaluate CodeCloak under typical app repackaging scenarios. Experimental results show that CodeCloak can effectively protect apps against repackaging attacks at the cost of minimum overhead.

Enhancing the Description-to-Behavior Fidelity in Android Apps with Privacy Policy

Since more than 96 percent of mobile malware targets the Android platform, various techniques based on static code analysis or dynamic behavior analysis have been proposed to detect malicious apps. As malware is becoming more complicated and stealthy, recent research proposed a promising detection approach that looks for the inconsistency between an app's permissions and its description. In this paper, we first revisit this approach and reveal that using description and permission will lead to many false positives because descriptions often fail to declare all sensitive operations. Then, we propose exploiting an app's privacy policy and its bytecode to enhance the malware detection based on description and permissions. It is non-trivial to automatically analyze privacy policy and perform the cross-verification among these four kinds of software artifacts including, privacy policy, bytecode, description, and permissions. To address these challenging issues, we first propose a novel data flow model for analyzing privacy policy, and then develop a new system, named TAPVerifier, for carrying out investigation of individual software artifacts and conducting the cross-verification. The experimental results show that TAPVerifier can analyze privacy policy with a high accuracy and recall rate. More importantly, integrating privacy policy and bytecode level information can remove up to 59.4 percent false alerts of the state-of-the-art systems, such as AutoCog, CHABADA, etc.

Search-Based Energy Optimization of Some Ubiquitous Algorithms

Reducing computational energy consumption is of growing importance, particularly at the extremes (i.e., mobile devices and datacentres). Despite the ubiquity of the Java virtual machine (JVM), very little work has been done to apply search-based software engineering (SBSE) to minimize the energy consumption of programs that run on it. We describe OPACITOR, a tool for measuring the energy consumption of JVM programs using a bytecode level model of energy cost. This has several advantages over timebased energy approximations or hardware measurements. It is 1) deterministic, 2) unaffected by the rest of the computational environment, 3) able to detect small changes in execution profile, making it highly amenable to metaheuristic search, which requires locality of representation. We show how generic SBSE approaches coupled with OPACITOR achieve substantial energy savings for three widely used software components. Multilayer perceptron implementations minimizing both energy and error were found, and energy reductions of up to 70% and 39.85% were obtained over the original code for Quicksort and object-oriented container classes, respectively. These highlight three important considerations for automatically reducing computational energy: tuning software to particular distributions of data; trading off energy use against functional properties; and handling internal dependencies that can exist within software that render simple sweeps over program variants sub-optimal. Against these, global search greatly simplifies the developer's job, freeing development time for other tasks.

DroidNative: Automating and optimizing detection of Android native code malware variants

According to the Symantec and F-Secure threat reports, mobile malware development in 2013 and 2014 has continued to focus almost exclusively (~99%) on the Android platform. Malware writers are applying stealthy mutations (obfuscations) to create malware variants, thwarting detection by signature-based detectors. In addition, the plethora of more sophisticated detectors making use of static analysis techniques to detect such variants operate only at the bytecode level, meaning that malware embedded in native code goes undetected. A recent study shows that 86% of the most popular Android applications contain native code, making native code malware a plausible threat vector. This paper proposes DroidNative, an Android malware detector that uses specific control flow patterns to reduce the effect of obfuscations and provides automation. As far as we know, DroidNative is the first system that builds cross-platform (x86 and ARM) semantic-based signatures at the Android native code level, allowing the system to detect malware embedded in either bytecode or native code. When tested with a dataset of 5490 samples, DroidNative achieves a detection rate (DR) of 93.57% and a false positive rate of 2.7%. When tested with traditional malware variants, it achieves a DR of 99.48%, compared to the DRs of academic and commercial tools that range from 8.33% to 93.22%.

Control flow obfuscation for Android applications

Android apps are vulnerable to reverse engineering, which makes app tampering and repackaging relatively easy. While obfuscation is widely known to make reverse engineering harder, complex and effective control flow obfuscations by rearranging Android bytecode instructions have not been implemented in various Android obfuscation tools. This paper presents our control-flow obfuscation techniques for Android apps at the Dalvik bytecode level. Our three proposed schemes go beyond simple control-flow transformations employed by existing Android obfuscators, and make it difficult for static analysis to determine the actual app control flows. To realize this, we also address a previously-unsolved register-type conflict problem that can be raised by the verifier module of the Android runtime system by means of a type separation technique. Our analysis and experimentation show that the schemes can offer effective obfuscation with reasonable performance and size overheads. Combined with the existing data and layout obfuscation techniques, our schemes can offer attractive measures to hinder reverse engineering and code analysis on Android apps, and help safeguard Android app developers' heavy investment in their apps.

Accurate profiling in the presence of dynamic compilation

Many profilers based on bytecode instrumentation yield wrong results in the presence of an optimizing dynamic compiler, either due to not being aware of optimizations such as stack allocation and method inlining, or due to the inserted code disrupting such optimizations. To avoid such perturbations, we present a novel technique to make any profiler implemented at the bytecode level aware of optimizations performed by the dynamic compiler. We implement our approach in a state-of-the-art Java virtual machine and demonstrate its significance with concrete profilers. We quantify the impact of escape analysis on allocation profiling, object life-time analysis, and the impact of method inlining on callsite profiling. We illustrate how our approach enables new kinds of profilers, such as a profiler for non-inlined callsites, and a testing framework for locating performance bugs in dynamic compiler implementations.

Metamorphic code generation from LLVM bytecode

Metamorphic software changes its internal structure across generations with its functionality remaining unchanged. Metamorphism has been employed by malware writers as a means of evading signature detection and other advanced detection strategies. However, code morphing also has potential security benefits, since it can serve to increase the “genetic diversity” of software. We have created a metamorphic code generator within the LLVM compiler framework. LLVM is a three-phase compiler that supports multiple source languages and target architectures. It uses a common intermediate representation (IR) bytecode in its optimizer. Consequently, any supported high-level programming language is transformed to this IR bytecode as part of the LLVM compilation process. Our metamorphic generator functions at the IR bytecode level, which provides many advantages over morphing at the assembly or source code level. The morphing techniques that we employ include dead code insertion and transposition, where the dead code is actually executed within the morphed code, making its detection and removal more challenging. We have verified the effectiveness of our code morphing using hidden Markov model analysis.

Towards a General Framework for Formal Reasoning about Java Bytecode Transformation

Program transformation has gained a wide interest since it is used for several purposes: altering semantics of a program, adding features to a program or performing optimizations. In this paper we focus on program transformations at the bytecode level. Because these transformations may introduce errors, our goal is to provide a formal way to verify the update and establish its correctness. The formal framework presented includes a definition of a formal semantics of updates which is the base of a static verification and a scheme based on Hoare triples and weakest precondition calculus to reason about behavioral aspects in bytecode transformation

JEqualityGen

Manually implementing equals (for object comparisons) and hashCode (for object hashing) methods in large software projects is tedious and error-prone. This is due to many special cases, such as field shadowing, comparison between different types, or cyclic object graphs. Here, we present JEqualityGen, a source code generator that automatically derives implementations of these methods. JEqualityGen proceeds in two states: it first uses source code reflection in MetaAspectJ to generate aspects that contain the method implementations, before it uses weaving on the bytecode level to insert these into the target application. JEqualityGen generates not only correct, but efficient source code that on a typical large-scale Java application exhibits a performance improvement of more than two orders of magnitude in the equality operations generated, compared to an existing system based on runtime reflection. JEqualityGen achieves this by generating runtime profiling code that collects data. This enables it to generate optimised method implementations in a second round.

ADVANCED AND RAPID DEVELOPMENT OF DYNAMIC ANALYSIS TOOLS FOR JAVA

Low-level bytecode instrumentation techniques are widely used in many software-engineering tools for the Java Virtual Machine (JVM), that perform some form of dynamic program analysis, such as profilers or debuggers. While program manipulation at the bytecode level is very flexible, because the possible bytecode transformations are not restricted, tool development based on this technique is tedious and error-prone. As a promising alternative, the specification of bytecode instrumentation at a higher level using aspect-oriented programming (AOP) can reduce tool development time and cost. Unfortunately, prevailing AOP frameworks lack some features that are essential for certain dynamic analyses. In this article, we focus on three common shortcomings in AOP frameworks with respect to the development of aspect-based tools - (1) the lack of mechanisms for passing data between woven advices in local variables, (2) the support for user-defined static analyses at weaving time, and (3) the absence of pointcuts at the level of individual basic blocks of code. We propose @J, an annotation-based AOP language and weaver that integrates support for these three features. The benefits of the proposed features are illustrated with concrete examples.

Worst‐case execution time analysis for a Java processor

AbstractIn this paper, we propose a solution for a worst‐case execution time (WCET) analyzable Java system: a combination of a time‐predictable Java processor and a tool that performs WCET analysis at Java bytecode level. We present a Java processor, called JOP, designed for time‐predictable execution of real‐time tasks. The execution time of bytecodes, the instructions of the Java virtual machine, is known to cycle accuracy for JOP. Therefore, JOP simplifies the low‐level WCET analysis. A method cache, which fills whole Java methods into the cache, simplifies cache analysis. The WCET analysis tool is based on integer linear programming. The tool performs the low‐level analysis at the bytecode level and integrates the method cache analysis. An integrated data‐flow analysis performs receiver‐type analysis for dynamic method dispatches and loop‐bound analysis. Furthermore, a model checking approach to WCET analysis is presented where the method cache can be exactly simulated. The combination of the time‐predictable Java processor and the WCET analysis tool is evaluated with standard WCET benchmarks and three real‐time applications. The WCET friendly architecture of JOP and the integrated method cache analysis yield tight WCET bounds. Comparing the exact, but expensive, model checking‐based analysis of the method cache with the static approach demonstrates that the static approximation of the method cache is sufficiently tight for practical purposes. Copyright © 2010 John Wiley &amp; Sons, Ltd.

Enhancing source-level programming tools with an awareness of transparent program transformations

Programs written in managed languages are compiled to a platform-independent intermediate representation, such as Java bytecode. The relative high level of Java bytecode has engendered a widespread practice of changing the bytecode directly, without modifying the maintained version of the source code. This practice, called bytecode engineering or enhancement , has become indispensable in introducing various concerns, including persistence, distribution, and security, transparently. For example, transparent persistence architectures help avoid the entanglement of business and persistence logic in the source code by changing the bytecode directly to synchronize objects with stable storage. With functionality added directly at the bytecode level, the source code reflects only partial semantics of the program. Specifically, the programmer can neither ascertain the program's runtime behavior by browsing its source code, nor map the runtime behavior back to the original source code. This paper presents an approach that improves the utility of source-level programming tools by providing enhancement specifications written in a domain-specific language. By interpreting the specifications, a source-level programming tool can gain an awareness of the bytecode enhancements and improve its precision and usability. We demonstrate the applicability of our approach by making a source code editor and a symbolic debugger enhancements-aware.

J-Orchestra

J-Orchestra is a system that enhances centralized Java programs with distribution capabilities. Operating at the bytecode level, J-Orchestra transforms a centralized Java program (i.e., running on a single Java Virtual Machine (JVM)) into a distributed one (i.e., running across multiple JVMs). This transformation effectively separates distribution concerns from the core functionality of a program. J-Orchestra follows a semiautomatic transformation process. Through a GUI, the user selects program elements (at class granularity) and assigns them to network locations. Based on the user's input, the J-Orchestra backend automatically partitions the program through compiler-level techniques, without changes to the JVM or to the Java Runtime Environment (JRE) classes. By means of bytecode engineering and code generation, J-Orchestra substitutes method calls with remote method calls, direct object references with proxy references, etc. It also translates Java language features (e.g., static methods and fields, inheritance, inner classes, new object construction, etc.) for efficient distributed execution. We detail the main technical issues that J-Orchestra addresses, including its mechanism for program transformation in the presence of unmodifiable code (e.g., in JRE classes) and the translation of concurrency and synchronization constructs to work correctly over the network. We further discuss a case study of transforming a large, commercial, third-party application for efficient execution in a client server environment and outline the architectural characteristics of centralized programs that are amenable to automated distribution with J-Orchestra.