SPECjvm98 Benchmarks Research Articles

Skip compactors for garbage collection based on space utilization

ABSTRACTThe mark-compact garbage collection algorithm shifts live objects so that they are next to each other to maximize space utilization efficiency and enable rapid object allocation. However, the algorithm requires multiple passes over a heap of objects, extending the time required for reclaiming unused memory. To shorten the collection time, we propose a skip compactor for garbage collection algorithm that avoids accessing dead objects by merging contiguous dead objects and reduces the number of object relocations. In this paper, we have implemented the proposed algorithm in the Jikes RVM, and measured the performance with the SPECjvm98 and DaCapo2006 benchmark suites. The proposed algorithm reduces the pause time of programs by 25% on average.

Barriers reconsidered, friendlier still!

Read and write barriers mediate access to the heap allowing the collector to control and monitor mutator actions. For this reason, barriers are a powerful tool in the design of any heap management algorithm, but the prevailing wisdom is that they impose significant costs. However, changes in hardware and workloads make these costs a moving target. Here, we measure the cost of a range of useful barriers on a range of modern hardware and workloads. We confirm some old results and overturn others. We evaluate the microarchitectural sensitivity of barrier performance and the differences among benchmark suites. We also consider barriers in context, focusing on their behavior when used in combination, and investigate a known pathology and evaluate solutions. Our results show that read and write barriers have average overheads as low as 5.4% and 0.9% respectively. We find that barrier overheads are more exposed on the workload provided by the modern DaCapo benchmarks than on old SPECjvm98 benchmarks. Moreover, there are differences in barrier behavior between in-order and out-of- order machines, and their respective memory subsystems, which indicate different barrier choices for different platforms. These changing costs mean that algorithm designers need to reconsider their design choices and the nature of their resulting algorithms in order to exploit the opportunities presented by modern hardware.

Heuristic optimisation algorithm for Java dynamic compilation

Dynamic compilation increases Java virtual machine (JVM) performance because running compiled codes is faster than interpreting Java bytecodes. However, inappropriate decision on dynamic compilation may degrade performance owing to compilation overhead. A good heuristic algorithm for dynamic compilation should achieve an appropriate balance between compilation overhead and performance gain in each method invocation sequence. A method-size and execution-time heuristic algorithm is proposed in the study. The key principle of the algorithm is that different method-sizes necessitate different compile thresholds for optimal performance. A parameter search mechanism using a genetic algorithm for dynamic compilation is proposed to find optimised multi-thresholds in the algorithm. This heuristic algorithm is evaluated in an openJDK Java Server JVM using SPEC JVM98 benchmark suite. The algorithm shows an overall advantage in performance speedup when testing benchmarks and gain speedup by 19.1% on average. The algorithm also increases the performance of original openJDK by 10.2% when extended to the whole benchmark suite.

Skew-space garbage collection

Semispace garbage collectors relocate all the live objects in one step, which is simple and leads to good performance. Compared with mark-compact collectors, however, they need to reserve extra heap space for copying live objects. As much as half of the heap could be reserved as it is possible that all the allocated objects survive. In reality, however, most programs exhibit a high infant mortality, and therefore reserving half the heap is wasteful.We have observed that the memory usage of many ordinary programs is relatively stable over the course of their execution. This provides an opportunity for online predictions to dynamically adjust and optimize the reserved space. Consequently, we propose a skew-space garbage collector that reserves space dynamically. This collector is implemented using the MMTk framework of the Jikes RVM and gives encouraging results against related garbage collection algorithms for the DaCapo and SPECjvm98 benchmarks.

An efficient approach to cyclic reference counting based on a coarse-grained search

Reference counting is known to have problems working with cyclic structures. In this paper, we present an efficient approach to cyclic reference counting, consisting of two key components. The first is a coarse-grained cycle collection algorithm that essentially performs a coarser (lightweight) analysis of the computation graph and thus greatly reduces the tracing cost (in comparison with the algorithms based on trial deletion to detect cycles). Our new cycle collector relies on this algorithm to obtain efficiency. Second, a predefined backup algorithm is incorporated to eliminate a theoretical problem that appears in the coarse-grained algorithm, thereby making the collector more practical. In this regard, we develop a heuristic based on the runtime behavior of the cycle collection to help the collector determine when to trigger the backup one. We have implemented and evaluated the proposed cycle collector on the Jikes RVM, where the SPECjvm98 benchmarks were applied. The results demonstrate that the novel approach is efficient and practical, compared to a modern cycle collector based on trial deletion.

Array bounds check elimination in the context of deoptimization

Whenever an array element is accessed, Java virtual machines execute a compare instruction to ensure that the index value is within the valid bounds. This reduces the execution speed of Java programs. Array bounds check elimination identifies situations in which such checks are redundant and can be removed. We present an array bounds check elimination algorithm for the Java HotSpot™ VM based on static analysis in the just-in-time compiler. The algorithm works on an intermediate representation in static single assignment form and maintains conditions for index expressions. It fully removes bounds checks if it can be proven that they never fail. Whenever possible, it moves bounds checks out of loops. The static number of checks remains the same, but a check inside a loop is likely to be executed more often. If such a check fails, the executing program falls back to interpreted mode, avoiding the problem that an exception is thrown at the wrong place. The evaluation shows a speedup near to the theoretical maximum for the scientific SciMark benchmark suite and also significant improvements for some Java Grande benchmarks. The algorithm slightly increases the execution speed for the SPECjvm98 benchmark suite. The evaluation of the DaCapo benchmarks shows that array bounds checks do not have a significant impact on the performance of object-oriented applications.

Design of the Java HotSpot™ client compiler for Java 6

Version 6 of Sun Microsystems' Java HotSpot™ VM ships with a redesigned version of the client just-in-time compiler that includes several research results of the last years. The client compiler is at the heart of the VM configuration used by default for interactive desktop applications. For such applications, low startup and pause times are more important than peak performance. This paper outlines the new architecture of the client compiler and shows how it interacts with the VM. It presents the intermediate representation that now uses static single-assignment (SSA) form and the linear scan algorithm for global register allocation. Efficient support for exception handling and deoptimization fulfills the demands that are imposed by the dynamic features of the Java programming language. The evaluation shows that the new client compiler generates better code in less time. The popular SPECjvm98 benchmark suite is executed 45% faster, while the compilation speed is also up to 40% better. This indicates that a carefully selected set of global optimizations can also be integrated in just-in-time compilers that focus on compilation speed and not on peak performance. In addition, the paper presents the impact of several optimizations on execution and compilation speed. As the source code is freely available, the Java HotSpot™ VM and the client compiler are the ideal basis for experiments with new feedback-directed optimizations in a production-level Java just-in-time compiler. The paper outlines research projects that add fast algorithms for escape analysis, automatic object inlining, and array bounds check elimination.

Generating Java Compiler Optimizers Using Bidirectional CTL

There have been several research works that analyze and optimize programs using temporal logic. However, no evaluation of optimization time or execution time of these implementations has been done for any real programming language. In this paper, we present a system that generates a Java optimizer from specifications in temporal logic. The specification is simpler, and the generated optimizers run more efficiently than previously reported work. We implemented a new model checker for a bidirectional CTL (computational tree logic) called CTLbd, which is equivalent to CTL-FV [Lacey, D., Jones, N.D., Van Wyk, E. and Frederiksen, C.C.: Compiler optimization correctness by temporal logic. Higher-Order and Symbolic Computation, Vol. 17, No. 3, pp. 173–206, 2004] after removing free variables. The model checker can check future and past temporal CTL operators symmetrically without any conversion. We also present a new specification language based on the bidirectional CTL that can express typical optimization rules very naturally. By adding rewriting conditions to allow for temporary variables and considering real-world language features such as exceptions, the system can perform optimization of Java programs. So far, a compiler optimizer using temporal logic was assumed to be impractical, because it consumes too much time. However, with our method, the generated Java compiler optimizer can compile seven of the SPECjvm98 benchmarks with a compile time from 4 seconds to 4 minutes.

Java heap protection for debugging native methods

Java virtual machine (JVM) crashes are often due to an invalid memory reference to the JVM heap. Before the bug that caused the invalid reference can be fixed, its location must be identified. It can be in either the JVM implementation or the native library written in C invoked from Java applications. To help system engineers identify the location, we implemented a feature using page protection that prevents threads executing native methods from referring to the JVM heap. This feature protects the JVM heap during native method execution; if the heap is referred to invalidly, it interrupts the execution by generating a page-fault exception. It then reports the location where the exception was generated. The runtime overhead for using this feature depends on the frequency of native method calls because the protection is switched on each time a native method is called. We evaluated the runtime overhead by running the SPECjvm98, SPECjbb2000, VolanoMark, and JFCMark benchmark suites on a PC with two Intel Xeon ® 1.6 GHz processors. The performance loss was less than 2% for the benchmark items that do not call native methods so frequently (∼10 4 times per second) and 5%–20% for the benchmark items that do (10 4–10 5 times per second). The worst performance loss was 54%, which was recorded for a benchmark item that calls native methods 2.0×10 6 times per second.

Dynamic Inlining Based on Continuous Monitoring

Dynamic compilation is an effective optimization, but the overhead is too heavy, the accuracy of information is not enough, and the redundant code is growing rapidly in the current information collection and continuous monitoring. In this paper, the dynamic compiler is designed based on online feedback and continuous monitoring in Intel ORP (open runtime platform). To solve these problems in information collection, the program instrumentation technique is improved. The continuous monitoring for the type of virtual method's receiver object is implemented. Based on the information from online feedback and continuous monitor, the dynamic inlining optimization is invoked by the compiler. The dynamic unloading algorithm that releases the redundant code in the dynamic optimization is introduced. The results of SpecJVM98 and Java Grande Forum Benchmark show that the performance of JVM is improved on average and the system load is reduced by the dynamic unloading algorithm.

Method-specific dynamic compilation using logistic regression

Determining the best set of optimizations to apply to a program has been a long standing problem for compiler writers. To reduce the complexity of this task, existing approaches typically apply the same set of optimizations to all procedures within a program, without regard to their particular structure. This paper develops a new method-specific approach that automatically selects the best optimizations on a per method basis within a dynamic compiler. Our approach uses the machine learning technique of logistic regression to automatically derive a predictive model that determines which optimizations to apply based on the features of a method. This technique is implemented in the Jikes RVM Java JIT compiler. Using this approach we reduce the average total execution time of the SPECjvm98 benchmarks by 29%. When the same heuristic is applied to the DaCapo+ benchmark suite, we obtain an average 33% reduction over the default level O2 setting.

Leveraging disposable instrumentation to reduce coverage collection overhead

AbstractTesters use coverage data for test suite quality assessment, stopping criteria definition, and effort allocation. However, as the complexity of products and testing processes increases, the cost of coverage data collection may grow significantly, jeopardizing its potential application. To mitigate this problem this paper presents the concept of ‘disposable coverage instrumentation’—coverage instrumentation that is removed after its execution—through two techniques: local disposal, and collective disposal. A Java virtual machine was extended to support these techniques, and their potential is shown through two studies utilizing the Specjvm98 and Specjbb2000 benchmarks. The results indicate that the techniques can reduce coverage collection overhead by an order of magnitude over state‐of‐the‐art techniques. Copyright © 2006 John Wiley & Sons, Ltd.

Method-level phase behavior in java workloads

Java workloads are becoming more and more prominent on various computing devices. Understanding the behavior of a Java workload which includes the interaction between the application and the virtual machine (VM), is thus of primary importance during performance analysis and optimization. Moreover, as contemporary software projects are increasing in complexity, automatic performance analysis techniques are indispensable. This paper proposes an off-line method-level phase analysis approach for Java workloads that consists of three steps. In the first step, the execution time is computed for each method invocation. Using an off-line tool, we subsequently analyze the dynamic call graph (that is annotated with the method invocations' execution times) to identify method-level phases. Finally, we measure performance characteristics for each of the selected phases. This is done using hardware performance monitors. As such, our approach allows for linking microprocessor-level information at the individual methods in the Java application's source code. This is extremely interesting information during performance analysis and optimization as programmers can use this information to optimize their code. We evaluate our approach in the Jikes RVM on an IA-32 platform using the SPECjvm98 and SPECjbb2000 benchmarks. This is done according to a number of important criteria: the overhead during profiling, the variability within and between the phases, its applicability in Java workload characterization (measuring performance characteristics of the various VM components) and application bottleneck identification.

When and what to compile/optimize in a virtual machine?

To speed up the computation of some of the object languages, virtual machines use dynamic compilation. But due to compilation taking place during user application runtime, one has to choose which methods to compile, so that the compile time has little impact on the total runtime of the actual application. In order to achieve this, we have to use an estimate to determine the process of compilation. Current virtual machines use run time information such as number of calls made to this method, size of the method and as well as number of times the back edges taken to determine the hotness of a method. They use this information against some threshold which does not have any relationship with the method that is being considered for compilation.Detecting the hot routines is very important from the perspective of performance since optimized methods run 10 times faster than the interpreted version. In this paper we propose a new technique called relative estimation for determining the hotness of the functions. For each method in the application, we estimate the cost/benefits for compilation or optimization by doing method analysis. We select the methods for compilation or optimization initially based on the above metrics using the proposed relative estimation technique. Later on we use the online profile information such as backedge counters and invocation counts to adjust the relative estimation process itself. We have performed experiments to validate the effectiveness of the proposed method on SPECjvm98 benchmark suite and found that up to 4% performance improvement in the application execution time occurs.

How java programs interact with virtual machines at the microarchitectural level

Java workloads are becoming increasingly prominent on various platforms ranging from embedded systems, over general-purpose computers to high-end servers. Understanding the implications of all the aspects involved when running Java workloads, is thus extremely important during the design of a system that will run such workloads. In other words, understanding the interaction between the Java application, its input and the virtual machine it runs on, is key to a succesful design. The goal of this paper is to study this complex interaction at the microarchitectural level, e.g., by analyzing the branch behavior, the cache behavior, etc. This is done by measuring a large number of performance characteristics using performance counters on an AMD K7 Duron microprocessor. These performance characteristics are measured for seven virtual machine configurations, and a collection of Java benchmarks with corresponding inputs coming from the SPECjvm98 benchmark suite, the SPECjbb2000 benchmark suite, the Java Grande Forum benchmark suite and an open-source raytracer, called Raja with 19 scene descriptions. This large amount of data is further analyzed using statistical data analysis techniques, namely principal components analysis and cluster analysis. These techniques provide useful insights in an understandable way.From our experiments, we conclude that (i) the behavior observed at the microarchitectural level is primarily determined by the virtual machine for small input sets, e.g., the SPECjvm98 s1 input set; (ii) the behavior can be quite different for various input sets, e.g., short-running versus long-running benchmarks; (iii) for long-running benchmarks with few hot spots, the behavior can be primarily determined by the Java program and not the virtual machine, i.e., all the virtual machines optimize the hot spots to similarly behaving native code; (iv) in general, the behavior of a Java application running on one virtual machine can be significantly different from running on another virtual machine. These conclusions warn researchers working on Java workloads to be careful when using a limited number of Java benchmarks or virtual machines since this might lead to biased conclusions.

Data size optimizations for java programs

We present a set of techniques for reducing the memory consumption of object-oriented programs. These techniques include analysis algorithms and optimizations that use the results of these analyses to eliminate fields with constant values, reduce the sizes of fields based on the range of values that can appear in each field, and eliminate fields with common default values or usage patterns. We apply these optimizations both to fields declared by the programmer and to implicit fields in the runtime object header. Although it is possible to apply these techniques to any object-oriented program, we expect they will be particularly appropriate for memory-limited embedded systems.We have implemented these techniques in the MIT FLEX compiler system and applied them to the programs in the SPECjvm98 benchmark suite. Our experimental results show that our combined techniques can reduce the maximum live heap size required for the programs in our benchmark suite by as much as 40%. Some of the optimizations reduce the overall execution time; others may impose modest performance penalties.

Partitioned instruction cache architecture for energy efficiency

The demand for high-performance architectures and powerful battery-operated mobile devices has accentuated the need for low-power systems. In many media and embedded applications, the memory system can consume more than 50% of the overall system energy, making it a ripe candidate for optimization. To address this increasingly important problem, this article studies energy-efficient cache architectures in the memory hierarchy that can have a significant impact on the overall system energy consumption.Existing cache optimization approaches have looked at partitioning the caches at the circuit level and enabling/disabling these cache partitions (subbanks) at the architectural level for both performance and energy. In contrast, this article focuses on partitioning the cache resources architecturally for energy and energy-delay optimizations. Specifically, we investigate ways of splitting the cache into several smaller units, each of which is a cache by itself (called a subcache ). Subcache architectures not only reduce the per-access energy costs, but can potentially improve the locality behavior as well.The proposed subcache architecture employs a page-based placement strategy, a dynamic page remapping policy, and a subcache prediction policy in order to improve the memory system energy behavior, especially on-chip cache energy. Using applications from the SPECjvm98 and SPEC CPU2000 benchmarks, the proposed subcache architecture is shown to be very effective in improving both the energy and energy-delay metrics. It is more beneficial in larger caches as well.

Estimating internal memory fragmentation for Java programs

Dynamic memory management has been an important part of a large class of computer programs and with the recent popularity of object oriented programming languages, more specifically Java, high performance dynamic memory management algorithms continue to be of great importance. In this paper, an analysis of Java programs, provided by the SPECjvm98 benchmark suite, and their behavior, as this relates to fragmentation, is performed. Based on this analysis, a new model is proposed which allows the estimation of the total internal fragmentation that Java systems will incur prior to the programs execution. The proposed model can also accommodate any variation of segregated lists implementation. A comparison with a previously introduced fragmentation model is performed as well as a comparison with actual fragmentation values that were extracted from SPECjvm98. Finally the idea of a test-bed application that will use the proposed model to provide to programmers/developers the ability to know, prior to a programs execution, the fragmentation and memory utilization of their programs, is also introduced. With this application at hand developers as well as designers of applications could better assess the stability, efficiency as well reliability of their applications at compile time.

An adaptive, region-based allocator for java

This paper introduces an adaptive, region-based allocator for Java. The basic idea is to allocate non-escaping objects in local regions, which are allocated and freed in conjunction with their associated stack frames. By releasing memory associated with these stack frames, the burden on the garbage collector is reduced, possibly resulting in fewer collections.The novelty of our approach is that it does not require static escape analysis, programmer annotations, or special type systems. The approach is transparent to the Java programmer and relatively simple to add to an existing JVM. The system starts by assuming that all allocated objects are local to their stack region, and then catches escaping objects via write barriers. When an object is caught escaping, its associated allocation site is marked as a non-local site, so that subsequent allocations will be put directly in the global region. Thus, as execution proceeds, only those allocation sites that are likely to produce non-escaping objects are allocated to their local stack region.The paper presents the overall idea, and then provides details of a specific design and implementation. In particular, we present a region-based allocator and the necessary modifications of the Jikes RVM baseline JIT and a copying collector. Our experimental study evaluates the idea using the SPEC JVM98 benchmarks, plus one other large benchmark. We show that a region-based allocator is a reasonable choice, that overheads can be kept low, and that the adaptive system is successful at finding local regions that contain no escaping objects.

Estimating the impact of heap liveness information on space consumption in Java

We study the potential impact of different kinds of liveness information on the space consumption of a program in a garbage collected environment, specifically for Java. The idea is to measure the time difference between the actual time an object is collected by the garbage collector (GC) and the potential earliest time an object could be collected assuming liveness information were available. We focus on the following kinds of liveness information: (i) stack-reference liveness (local reference variable liveness in Java), (ii) global-reference liveness (static reference variable liveness in Java), (iii) heap-reference liveness (instance reference variable liveness or array reference liveness in Java), and (vi) any combination of (i)-(iii). We also provide some insights on the kind of interface between a compiler and GC that could achieve these potential savings.The Java Virtual Machine (JVM) was instrumented to measure (dynamic) liveness information. Experimental results are given for 10 benchmarks, including 5 of the SPEC-jvm98 benchmark suite. We show that in general stack-reference liveness may yield small benefits, global-reference liveness combined with stack-reference liveness may yield medium benefits, and heap-reference liveness yields the largest potential benefit. Specifically, for heap-reference liveness we measure an average potential savings of 39% using an interface with complete liveness information, and an average savings of 15% using a more restricted interface.