Real-time Garbage Collection Research Articles

Thread Resumption Barrier for Real-Time Garbage Collection in Multi-Threaded Environment

Thread Resumption Barrier for Real-Time Garbage Collection in Multi-Threaded Environment

Hierarchical real-time garbage collection

Memory management is a critical issue for correctness and performance in real-time embedded systems. Recent work on real-time garbage collectors has shown that it is possible to provide guarantees on worst-case pause times and minimum mutator utilization time. This paper presents a new hierarchical real-time garbage collection algorithm for mixed-priority and mixed-criticality environments. With hierarchical garbage collection, real-time programmers can partition the heap into a number of heaplets and for each partition choose to run a separate collector with a schedule that matches the allocation behavior and footprint of the real-time task using it. This approach lowers worst-case response times of real-time applications by 26%, while almost doubling mutator utilization -- all with only minimal changes to the application code.

Eventrons

While real-time garbage collection has achieved worst-case latencies on the order of a millisecond, this technology is approaching its practical limits. For tasks requiring extremely low latency, and especially periodic tasks with frequencies above 1 KHz, Java programmers must currently resort to the NoHeapRealtimeThread construct of the Real-Time Specification for Java. This technique requires expensive run-time checks, can result in unpredictable low-level exceptions, and inhibits communication with the rest of the garbage-collected application. We present Eventrons , a programming construct that can arbitrarily preempt the garbage collector, yet guarantees safety and allows its data to be visible to the garbage-collected heap. Eventrons are a strict subset of Java, and require no run-time memory access checks. Safety is enforced using a data-sensitive analysis and simple run-time support with extremely low overhead. We have implemented Eventrons in IBM's J9 Java virtual machine, and present experimental results in which we ran Eventrons at frequencies up to 22 KHz (a 45 μ s period). Across 10 million periods, 99.997% of the executions ran within 10 μ ss of their deadline, compared to 99.999% of the executions of the equivalent program written in C.

BIT: A Very Compact Scheme System for Microcontrollers

We present a compact implementation of Scheme for microcontrollers that includes a real-time garbage collector. The compiler runs on a normal workstation and produces byte-code from the source program. A smart linker links the byte-code with the runtime module. We demonstrate that with this system it is clearly possible to run realistic Scheme programs on a microcontroller with as little as 3 to 4 KB of RAM. Programs that access the whole Scheme library require only 13 KB of ROM. As a byproduct of this research, we designed a novel space-efficient real-time GC algorithm.

Static determination of allocation rates to support real-time garbage collection

While it is generally accepted that garbage-collected languages offer advantages over languages in which objects must be explicitly deallocated, real-time developers are leery of the adverse effects a garbage collector might have on real-time performance. Semiautomatic approaches based on regions have been proposed, but incorrect usage could cause unbounded storage leaks or program failure. Moreover, correct usage cannot be guaranteed at compile time. Recently, real-time garbage collectors have been developed that provide a guaranteed fraction of the CPU to the application, and the correct operation of those collectors has been proven, subject only to the specification of certain statistics related to the type and rate of objects allocated by the application. However, unless those statistics are provided or estimated appropriately, the collector may fail to collect dead storage at a rate sufficient to pace the application's need for storage. Overspecification of those statistics is safe but leaves the application with less than its possible share of the CPU, which may prevent the application from meeting its deadlines.In this paper we present a static analysis to bound conservatively an application's allocation rate . The analysis is based on a data flow framework that requires interprocedural evaluation. We present the framework and results from analyzing some Java benchmarks. Because static analysis is necessarily conservative, we also present measurements of our benchmarks' actual allocation rates.Our work is a necessary step toward making real-time garbage collectors attractive to the hard-real-time community. By guaranteeing a bound on statistics provided to a real-time collector, we can guarantee the operation of the collector for a given application.

Syncopation

Real-time garbage collection has been shown to be feasible, but for programs with high allocation rates, the utilization achievable is not sufficient for some systems.Since a high allocation rate is often correlated with a more high-level, abstract programming style, the ability to provide good real-time performance for such programs will help continue to raise the level of abstraction at which real-time systems can be programmed.We have developed techniques that allow generational collection to be used despite the problems caused by variance in program behavior over the short time scales in which a nursery can be collected. Syncopation allows such behavior to be detected by the scheduler in time for allocation to by-pass the nursery and allow real-time bounds to be met.We have provided an analysis of the costs of both generational and non-generational techniques, which allow the trade-offs to be evaluated quantitatively. We have also provided measurements of application behavior which show that while syncopation is necessary, the need for it is rare enough that generational collection can provide major improvements in real-time utilization. An additional technique, arraylet pre-tenuring , often significantly improves generational behavior.

Real-time garbage collection for flash-memory storage systems of real-time embedded systems

Flash-memory technology is becoming critical in building embedded systems applications because of its shock-resistant, power economic, and nonvolatile nature. With the recent technology breakthroughs in both capacity and reliability, flash-memory storage systems are now very popular in many types of embedded systems. However, because flash memory is a write-once and bulk-erase medium, we need a translation layer and a garbage-collection mechanism to provide applications a transparent storage service. In the past work, various techniques were introduced to improve the garbage-collection mechanism. These techniques aimed at both performance and endurance issues, but they all failed in providing applications a guaranteed performance. In this paper, we propose a real-time garbage-collection mechanism, which provides a guaranteed performance, for hard real-time systems. On the other hand, the proposed mechanism supports non-real-time tasks so that the potential bandwidth of the storage system can be fully utilized. A wear-leveling method, which is executed as a non-real-time service, is presented to resolve the endurance problem of flash memory. The capability of the proposed mechanism is demonstrated by a series of experiments over our system prototype.

A novel processor architecture with exact tag-free pointers

Along with the success of Java, garbage collection advances to embedded and real-time systems. In this paper a novel RISC processor architecture that lays the foundation for efficient real-time garbage collection in hardware and guarantees pointer integrity at the machine-code level is proposed. This article summarizes the most commonly used garbage collection methods and points out problems of contemporary software-based solutions to motivate architectural support for garbage collection.

On bounding time and space for multiprocessor garbage collection

Our paper described a real-time garbage collection algorithm for shared-memory multiprocessors. The work differed from previous work on multiprocessor garbage collection in that we were interested in placing hard bounds on pause times and memory usage. We felt this was important in justifying the term real-time, and for giving users a better handle on actual performance characteristics. Although the paper presented an algorithm without an implementation, and made some simplifying assumptions, we put considerable effort into making key aspects of the algorithm practical. In later work [9, 8] we implemented the ideas, and demonstrated performance characteristics that were in line with the characteristics predicted by the theory.This paper presents the first multiprocessor garbage collection algorithm with provable bounds on time and space. The algorithm is a real-time shared-memory copying colector. We prove that the algorithm requires at most 2( R (1 + 2/ k ) + N + 5 PD ) memory locations, where P is the number of processors, R is the maximum reachable space during a computation (number of locations accessible from the root set), N is the maximum number of reachable objects, D is the maximum depth of any data object, and k is a parameter specifying how many locations are copied each time a location is allocated. Furthermore we show that client threads are never stopped for more than time proportional to k non-blocking machine instructions. The bounds are guaranteed even with arbitrary length arrays. The collector only requires write-barriers (reads are unaffected by the collector), makes few assumptions about the threads that are generating the garbage, and allows them to run mostly asynchronously.

Memory Management for Real-Time Java: An Efficient Solution using Hardware Support*

This paper addresses the issue of improving the performance of memory management for real-time Java applications, building upon the real-time specification for Java (RTSJ) from the Real-Time Java Expert Group. In a first step, a collecting dynamic memory solution including both a real-time garbage collector and region-based memory management, is proposed. A thorough analysis of the parameters influencing the performance of write barriers in memory management, together with ways of improvement are then presented. Finally, the implementation of a memory management solution compliant with the RTSJ and integrating the proposed improvements is sketched.

Real-Time Garbage Collection for a Multithreaded Java Microcontroller

We envision the upcoming of microcontrollers and systems-on-chip that are based on multithreaded processor kernels due to the fast context switching ability of hardware multithreading. Moreover we envision an extensive market for Java-based applications in embedded real-time systems. This paper introduces two new garbage collection algorithms that are dedicated to real-time garbage collection on a multithreaded Java microcontroller. Our garbage collector runs in a thread slot in parallel to real-time applications. We show that our algorithms require only about 5–10% of the processor time for an effective garbage collection concerning our real-time benchmarks.

Controlling fragmentation and space consumption in the metronome, a real-time garbage collector for Java

Now that the use of garbage collection in languages like Java is becoming widely accepted due to the safety and software engineering benefits it provides, there is significant interest in applying garbage collection to hard real-time systems. Past approaches have generally suffered from one of two major flaws: either they were not provably real-time, or they imposed large space overheads to meet the real-time bounds.Our previous work [3] presented the Metronome, a mostly non-copying real-time collector. The Metronome achieves worst-case pause times of 6 milliseconds while maintaining consistent mutator CPU utilization rates of 50% with only 1.5-2.1 times the maximum heap space required by the application, which is comparable with space requirements for stop-the-world collectors.However, that algorithm assumed a constant collection rate, ignored program-dependent characteristics, and lacked a precise specification for when to trigger collection or how much defragmentation to perform. This paper refines the model by taking into account program properties such as pointer density, average object size, and locality of object size. This allows us to bound both the time for collection and consequently the space overhead required much more tightly. We show experimentally that most parameters usually are not subject to large variation, indicating that a small number of parameters will be sufficient to predict the time and space requirements accurately.Our previous work also did not present the details of our approach to avoiding and undoing fragmentation. In this paper we present a more detailed analysis of fragmentation than in previous work, and show how our collector is able to bound fragmentation to acceptable limits.

Parallel Generational Copying Garbage Collection Schemes for Shared-Memory Multiprocessors

In this paper, an improved parallel generational copying real-time garbage collection scheme for shared-memory multiprocessors, which supports load balancing among workers, has been proposed, implemented, and evaluated. The basic idea of improvement is developed from Ali's two papers [2,3]. The scheme proposed here is a form of copying collectors that attempt to eliminate the drawback of frequently copying long-lived (stable) objects. This class of schemes is called generational-based schemes, which is based on concentrating the collection efforts on small areas of memory, so-called young generation. They reduce the need for collecting the remaining large areas of memory, old generation. A modified scheme, without real-time response, has also been implemented and compared to the real-time one. A comparative study has been done for the two schemes with other two parallel non-generational copying garbage collection schemes founded by the author in [1]. According to this comparative study, we proved that the performance of generational schemes is better than the performance of non-generational schemes. Also, the overheads occurred due to real-time response have been calculated.

Active memory processor: a hardware garbage collector for real-time java embedded devices

Java possesses many advantages for embedded system development, including fast product deployment, portability, security, and a small memory footprint. As Java makes inroads into the market for embedded systems, much effort is being invested in designing real-time garbage collectors. The proposed garbage-collected memory module, a bitmap-based processor with standard DRAM cells is introduced to improve the performance and predictability of dynamic memory management functions that include allocation, reference counting, and garbage collection. As a result, memory allocation can be done in constant time and sweeping can be performed in parallel by multiple modules. Thus, constant time sweeping is also achieved regardless of heap size. This is a major departure from the software counterparts where sweeping time depends largely on the size of the heap. In addition, the proposed design also supports limited-field reference counting, which has the advantage of distributing the processing cost throughout the execution. However, this cost can be quite large and results in higher power consumption due to frequent memory accesses and the complexity of the main processor. By doing reference counting operation in a coprocessor, the processing is done outside of the main processor. Moreover, the hardware cost of the proposed design is very modest (about 8000 gates). Our study has shown that 3-bit reference counting can eliminate the need to invoke the garbage collector in all tested applications. Moreover, it also reduces the amount of memory usage by 77 percent.

A real-time garbage collector with low overhead and consistent utilization

Now that the use of garbage collection in languages like Java is becoming widely accepted due to the safety and software engineering benefits it provides, there is significant interest in applying garbage collection to hard real-time systems. Past approaches have generally suffered from one of two major flaws: either they were not provably real-time, or they imposed large space overheads to meet the real-time bounds. We present a mostly non-moving, dynamically defragmenting collector that overcomes both of these limitations: by avoiding copying in most cases, space requirements are kept low; and by fully incrementalizing the collector we are able to meet real-time bounds. We implemented our algorithm in the Jikes RVM and show that at real-time resolution we are able to obtain mutator utilization rates of 45% with only 1.6--2.5 times the actual space required by the application, a factor of 4 improvement in utilization over the best previously published results. Defragmentation causes no more than 4% of the traced data to be copied.

Joint scheduling of garbage collector and hard real-time tasks for embedded applications

Programs with complex data structures often require dynamic memory management based on automatic memory reclamation (garbage collection). A major problem in adopting garbage collection for embedded real-time systems is that it often causes unpredictable pauses and that, as a result of such delays, hard real-time tasks may miss their deadlines. In this paper, we propose a new real-time garbage collection technique for embedded applications. In our approach, the system jointly schedules garbage collector and hard real-time tasks using one of the aperiodic server approaches. Our study focuses on reducing memory requirements while guaranteeing the deadlines of hard real-time tasks. To achieve this objective, we model garbage collection requests as aperiodic hard real-time tasks, and schedule them using the sporadic server (SS). We also present an effective live-memory analysis to bound the worst-case garbage collection time. Performance analysis shows that the proposed approach considerably reduces the worst-case memory reservation compared with a background policy. The analytic results are verified by simulation based on trace-driven data.

A parallel, real-time garbage collector

We describe a parallel, real-time garbage collector and present experimental results that demonstrate good scalability and good real-time bounds. The collector is designed for shared-memory multiprocessors and is based on an earlier collector algorithm [2], which provided fixed bounds on the time any thread must pause for collection. However, since our earlier algorithm was designed for simple analysis, it had some impractical features. This paper presents the extensions necessary for a practical implementation: reducing excessive interleaving, handling stacks and global variables, reducing double allocation, and special treatment of large and small objects. An implementation based on the modified algorithm is evaluated on a set of 15 SML benchmarks on a Sun Enterprise 10000, a 64-way UltraSparc-II multiprocessor. To the best of our knowledge, this is the first implementation of a parallel, real-time garbage collector. The average collector speedup is 7.5 at 8 processors and 17.7 at 32 processors. Maximum pause times range from 3 ms to 5 ms. In contrast, a non-incremental collector (whether generational or not) has maximum pause times from 10 ms to 650 ms. Compared to a non-parallel, stop-copy collector, parallelism has a 39% overhead, while real-time behavior adds an additional 12% overhead. Since the collector takes about 15% of total execution time, these features have an overall time costs of 6% and 2%.

Optimal real-time garbage collection for acyclic pointer structures

A real-time garbage collector for acyclic pointer structures is derived. Operations on the structure, such as assignment, allocation and de-allocation, take worst-case constant time. With respect to this real-time behaviour, storage usage is minimal. A complete implementation is presented.

Live memory analysis for garbage collection in embedded systems

Real-time garbage collection is essential if object-oriented languages (in particular, Java) are to become predictable enough for real-time embedded systems. Although techniques for hard real-time garbage collection exist, they are based on estimations of the maximum amount of referenced (live) memory. Such estimations may be difficult to derive manually for complex programs.We present techniques for predicting the maximum amount of live memory in object-oriented languages with inheritance and virtual methods. Annotations are used to bound recursively defined data structures. The annotations may also be used for timing analysis of code traversing annotated structures.A prototype live memory analysis tool has been developed. The tool interactively provides predictions of the maximum amount of live memory referenced from an arbitrary reference or block in an object-oriented program.

Portable run-time type description for conventional compilers

Many useful programming language extensions and system support libraries require knowledge of the locations of fields within objects at run time. Examples include orthogonal persistent object stores, precise garbage collectors, data structure picklers, and parameter marshaling schemes.For clean and efficient implementation as libraries, these systems require run-time knowledge of in-memory layouts of data objects, which is unavailable in most traditionally compiled and linked programming languages, such as C, C++, and Ada. Even the recently standardized run-time type identification (RTTI) feature in C++ is insufficient, because it describes only language-level features of the type hierarchy and not the compiler-dependent object layout decisions.We present a facility for run-time type description , or RTTD, which extracts low-level layout information from debugging information generated by conventional compilers, and makes it available to user programs. We believe this to be the simplest and most portable approach to run-time type description, requiring no changes to existing compilers. In this paper, we describe the basic strategies and present details of our implementation for C++. We also sketch some extensions that we have implemented, including special treatment of C++'s virtual function table pointers to match persistent or foreign data objects with the actual code in a particular application.Our implementation of run-time type description is freely available. It is in regular use with multiple operating systems and compilers, in both free and commercial products, including a high-performance persistent object storage system for C++ and a real-time garbage collector.