Memory Usage Patterns Research Articles

Earth system modeling on modular supercomputing architecture: coupled atmosphere–ocean simulations with ICON 2.6.6-rc

Abstract. The confrontation of complex Earth system model (ESM) codes with novel supercomputing architectures poses challenges to efficient modeling and job submission strategies. The modular setup of these models naturally fits a modular supercomputing architecture (MSA), which tightly integrates heterogeneous hardware resources into a larger and more flexible high-performance computing (HPC) system. While parts of the ESM codes can easily take advantage of the increased parallelism and communication capabilities of modern GPUs, others lag behind due to the long development cycles or are better suited to run on classical CPUs due to their communication and memory usage patterns. To better cope with these imbalances between the development of the model components, we performed benchmark campaigns on the Jülich Wizard for European Leadership Science (JUWELS) modular HPC system. We enabled the weather and climate model Icosahedral Nonhydrostatic (ICON) to run in a coupled atmosphere–ocean setup, where the ocean and the model I/O is running on the CPU Cluster, while the atmosphere is simulated simultaneously on the GPUs of JUWELS Booster (ICON-MSA). Both atmosphere and ocean are running globally with a resolution of 5 km. In our test case, an optimal configuration in terms of model performance (core hours per simulation day) was found for the combination of 84 GPU nodes on the JUWELS Booster module to simulate the atmosphere and 80 CPU nodes on the JUWELS Cluster module, of which 63 nodes were used for the ocean simulation and the remaining 17 nodes were reserved for I/O. With this configuration the waiting times of the coupler were minimized. Compared to a simulation performed on CPUs only, the MSA approach reduces energy consumption by 45 % with comparable runtimes. ICON-MSA is able to scale up to a significant portion of the JUWELS system, making best use of the available computing resources. A maximum throughput of 170 simulation days per day (SDPD) was achieved when running ICON on 335 JUWELS Booster nodes and 268 Cluster nodes.

Study of Energy-Efficient Optimization Techniques for High-Level Homogeneous Resource Management

Resource management efficiency can be a beneficial step toward optimizing power consumption in software-hardware integrated systems. Languages such as C, C++, and Fortran have been extremely popular for dealing with optimization, memory management, and other resource management. We investigate novel algorithmic architectures capable of optimizing resource requirements and increasing energy efficiency. The experimental results obtained with C++ can be extended to other programming languages as well. We emphasize the inherent drawbacks of memory management operators. These operators are intended to be extremely generic in their application, just as the concept of dynamic memory is. As a result, they are unable to take advantage of the various optimization techniques and opportunities that specific use cases present. Each source code file is modeled after its own distinct memory usage pattern, which can be used to speed up memory management routines. Such concepts are frequently time-consuming and costly to implement; consequently, they are not the primary concern of application developers, as they require manual development and integration. We intend to address this gap by providing a suite of memory management algorithms that enable dramatic performance improvements at the source code level while allowing for seamless integration across multiple use cases. The techniques have been evaluated on several performance parameters, and results have been presented. In this paper, we have compared a variety of memory allocation techniques and compared their space and energy efficiency requirements. Three variants of SSDAM, SSDAM-E, and DLLOM strategies have been evaluated and compared against the base performance of new and delete operators. SSDAM-E, SSDAM with new delete operators, and DDLOM improve the memory consumption by the factors of 8.01, 7.0, and 4.0, respectively. In the worst case, SSDAM-E gave an average running time of 5.650 sec faster than the DLLOM average time of 7.496 sec. As far as energy efficiency is considered, SSDAM-Original and SSDAM-E-Original attain 100%, in comparison with the base efficiency of 12.48% characterized by new/delete operators.

Cloud-BlackBox: Toward practical recording and tracking of VM swarms for multifaceted cloud inspection

Given the widening scope of the utilization and application of cloud computing services from general to mission-critical systems such as strategic military, financial, and the information systems of governmental agencies, the need for the development of improved methods to ensure the stability and security of cloud data and services is being increasingly emphasized. Various approaches have been developed to improve the security and stability of cloud infrastructure. In particular, the continuous inspection of the memory of Virtual Machine (VM) instances in the cloud platform has been an important factor in identifying the causes of security incidents related to zero-day vulnerabilities and critical system faults in cloud infrastructure. However, despite numerous studies in the field of continuous memory inspection, it is difficult to find a practical solution that is deployable in commercial-off-the-shelf cloud platforms. For instance, continuous memory snapshots generally cause various problems such as increased VM downtime occurrences, user-obstructive latency for memory snapshots, VM performance degradation, and massive data generation. To alleviate these limitations, we propose Cloud-BlackBox, which enables the recording of the memory of VM swarms running on cloud platforms that require a very high level of stability and security, and facilitates the flexible analysis of the recorded memory on a large-scale. A VM swarm refers to an environment in which multiple VMs are run in parallel. The proposed Cloud-BlackBox method provides the following benefits. First, by clustering VM swarm kernel memory, the amount of computation required to capture memory snapshots and the size of the generated snapshot images are minimized. Further, we propose a mechanism to merge kernel memory by rapidly identifying the homogeneity of the memory layout through analysis of the underlying base image and introspection of the running VM. The application of the proposed mechanism led to a storage reduction by a factor of 12.85. Second, a cognitive-scale bitmap was designed to track changes in the memory of VM swarms. The cognitive-scale bitmap is a mechanism that can dynamically manage the tracking of memory change information by recognizing the memory usage patterns of component VMs. With the designed cognitive-scale bitmap, the time required for the collection of a memory snapshot was reduced by more than 14.85 times, and the VM input/output (I/O) performance degradation was reduced by 50%. Third, a synchronized accessible memory interchange (SAMI) mechanism is proposed to facilitate the agile in-depth analysis of large-scale memory resources. Cloud-BlackBox tracks and records memory change information. Therefore, a procedure for restoring the recorded memory to a raw-memory analyzable form is required to analyze the recorded memory. The SAMI mechanism assists the analyst in ensuring consistent memory restoration performance when arbitrarily selecting recorded memory. Furthermore, SAMI is useful for reducing the scope of analysis without memory restoration simply by analyzing recorded metadata. Consequently, the revised schemes inside Cloud-BlackBox have several applications in various fields, such as advanced detection of malicious activities, service error recovery, malware analysis, and antivirus functions. In addition, the proposed approach has been implemented on a campus-wide cloud computing service called SysCore-Cloud.

Machine-Learning Based Memory Prediction Model for Data Parallel Workloads in Apache Spark

A lack of memory can lead to job failures or increase processing times for garbage collection. However, if too much memory is provided, the processing time is only marginally reduced, and most of the memory is wasted. Many big data processing tasks are executed in cloud environments. When renting virtual resources in a cloud environment, it is necessary to pay the cost according to the specifications of resources (i.e., the number of virtual cores and the size of memory), as well as rental time. In this paper, given the type of workload and volume of the input data, we analyze the memory usage pattern and derive the efficient memory size of data-parallel workloads in Apache Spark. Then, we propose a machine-learning-based prediction model that determines the efficient memory for a given workload and data. To determine the validity of the proposed model, we applied it to data-parallel workloads which include a deep learning model. The predicted memory values were in close agreement with the actual amount of required memory. Additionally, the whole building time for the proposed model requires a maximum of 44% of the total execution time of a data-parallel workload. The proposed model can improve memory efficiency up to 1.89 times compared with the vanilla Spark setting.

Analysis of cache behaviour and software optimizations for faster on-chip network simulations

Fast simulations are critical in reducing time to market in chip multiprocessors and system-on-chips. Several simulators have been used to evaluate the performance and power consumed by network-on-chips (NoCs). To speedup the simulations, it is necessary to investigate and optimize the hotspots in the simulator source code. Among several simulators available, Booksim2.0 has been chosen for the experimentation as it is being extensively used in the NoC community. In this paper, the cache and memory system behavior of Booksim2.0 have been analyzed to accurately monitor input dependent performance bottlenecks. The measurements show that cache and memory usage patterns vary widely based on the input parameters given to Booksim2.0. Based on these measurements, the cache configuration having the least misses has been identified. To further reduce the cache misses, software optimization techniques such as removal of unused functions, loop interchanging and replacing post-increment operator with pre-increment operator for non-primitive data types have been employed. The cache misses were reduced by 18.52%, 5.34% and 3.91% by employing above technology respectively. Thread parallelization and vectorization have been employed to improve the overall performance of Booksim2.0. The OpenMP programming model and SIMD are used for parallelizing and vectorizing the more time-consuming portions of Booksim2.0. Speedups of 2.93× and 3.97× were observed for the Mesh topology with $$30\times 30$$ network size by employing thread parallelization and vectorization respectively.

An experimental evaluation of garbage collectors on big data applications

Popular big data frameworks, ranging from Hadoop MapReduce to Spark, rely on garbage-collected languages, such as Java and Scala. Big data applications are especially sensitive to the effectiveness of garbage collection (i.e., GC), because they usually process a large volume of data objects that lead to heavy GC overhead. Lacking in-depth understanding of GC performance has impeded performance improvement in big data applications. In this paper, we conduct the first comprehensive evaluation on three popular garbage collectors, i.e., Parallel, CMS, and G1, using four representative Spark applications. By thoroughly investigating the correlation between these big data applications' memory usage patterns and the collectors' GC patterns, we obtain many findings about GC inefficiencies. We further propose empirical guidelines for application developers, and insightful optimization strategies for designing big-data-friendly garbage collectors.

Realizing Memory-Optimized Distributed Graph Processing

A multitude of contemporary applications heavily involve graph data whose size appears to be ever–increasing. This trend shows no signs of subsiding and has caused the emergence of a number of distributed graph processing systems including Pregel , Apache Giraph , and GraphX . However, the unprecedented scale now reached by real-world graphs hardens the task of graph processing due to excessive memory demands even for distributed environments. By and large, such contemporary graph processing systems employ ineffective in-memory representations of adjacency lists. Therefore, memory usage patterns emerge as a primary concern in distributed graph processing. We seek to address this challenge by exploiting empirically-observed properties demonstrated by graphs generated by human activity. In this paper, we propose 1) three compressed adjacency list representations that can be applied to any distributed graph processing system, 2) a variable-byte encoded representation of out-edge weights for space-efficient support of weighted graphs, and 3) a tree-based compact out-edge representation that allows for efficient mutations on the graph elements. We experiment with publicly-available graphs whose size reaches two-billion edges and report our findings in terms of both space-efficiency and execution time. Our suggested compact representations do reduce respective memory requirements for accommodating the graph elements up–to 5 times if compared with state-of-the-art methods. At the same time, our memory-optimized methods retain the efficiency of uncompressed structures and enable the execution of algorithms for large scale graphs in settings where contemporary alternative structures fail due to memory errors.

Accelerating Application Start-up with Nonvolatile Memory in Android Systems

Application launch time in mobile systems is critical in many cases because it can adversely affect user experience. Android has employed several software techniques to reduce application launch time. For example, Android shares memory space among applications to reduce the loading time of libraries. It also keeps applications in memory, even after the applications are terminated, to reduce start-up time. However, not much research has been done from a hardware perspective to reduce application launch time. In this article, the authors analyze memory usage patterns of Android applications and suggest several hardware optimization techniques. They also demonstrate the benefit of using a phase-change memory such as nonvolatile memory to accelerate start-up time.

Symbiotic Dynamic Memory Balancing for Virtual Machines in Smart TV Systems

Smart TV is expected to bring cloud services based on virtualization technologies to the home environment with hardware and software support. Although most physical resources can be shared among virtual machines (VMs) using a time sharing approach, allocating the proper amount of memory to VMs is still challenging. In this paper, we propose a novel mechanism to dynamically balance the memory allocation among VMs in virtualized Smart TV systems. In contrast to previous studies, where a virtual machine monitor (VMM) is solely responsible for estimating the working set size, our mechanism is symbiotic. Each VM periodically reports its memory usage pattern to the VMM. The VMM then predicts the future memory demand of each VM and rebalances the memory allocation among the VMs when necessary. Experimental results show that our mechanism improves performance by up to 18.28 times and reduces expensive memory swapping by up to 99.73% with negligible overheads (0.05% on average).

Portable performance on heterogeneous architectures

Trends in both consumer and high performance computing are bringing not only more cores, but also increased heterogeneity among the computational resources within a single machine. In many machines, one of the greatest computational resources is now their graphics coprocessors (GPUs), not just their primary CPUs. But GPU programming and memory models differ dramatically from conventional CPUs, and the relative performance characteristics of the different processors vary widely between machines. Different processors within a system often perform best with different algorithms and memory usage patterns, and achieving the best overall performance may require mapping portions of programs across all types of resources in the machine. To address the problem of efficiently programming machines with increasingly heterogeneous computational resources, we propose a programming model in which the best mapping of programs to processors and memories is determined empirically. Programs define choices in how their individual algorithms may work, and the compiler generates further choices in how they can map to CPU and GPU processors and memory systems. These choices are given to an empirical autotuning framework that allows the space of possible implementations to be searched at installation time. The rich choice space allows the autotuner to construct poly-algorithms that combine many different algorithmic techniques, using both the CPU and the GPU, to obtain better performance than any one technique alone. Experimental results show that algorithmic changes, and the varied use of both CPUs and GPUs, are necessary to obtain up to a 16.5x speedup over using a single program configuration for all architectures.

Portable performance on heterogeneous architectures

Trends in both consumer and high performance computing are bringing not only more cores, but also increased heterogeneity among the computational resources within a single machine. In many machines, one of the greatest computational resources is now their graphics coprocessors (GPUs), not just their primary CPUs. But GPU programming and memory models differ dramatically from conventional CPUs, and the relative performance characteristics of the different processors vary widely between machines. Different processors within a system often perform best with different algorithms and memory usage patterns, and achieving the best overall performance may require mapping portions of programs across all types of resources in the machine. To address the problem of efficiently programming machines with increasingly heterogeneous computational resources, we propose a programming model in which the best mapping of programs to processors and memories is determined empirically. Programs define choices in how their individual algorithms may work, and the compiler generates further choices in how they can map to CPU and GPU processors and memory systems. These choices are given to an empirical autotuning framework that allows the space of possible implementations to be searched at installation time. The rich choice space allows the autotuner to construct poly-algorithms that combine many different algorithmic techniques, using both the CPU and the GPU, to obtain better performance than any one technique alone. Experimental results show that algorithmic changes, and the varied use of both CPUs and GPUs, are necessary to obtain up to a 16.5x speedup over using a single program configuration for all architectures.

Supports for transparent object-migration in PDES systems

It is well known that Parallel Discrete Event Simulation systems may suffer, in terms of delivered performance, from imbalance of the computational load. In case of conservative synchronization we may experience CPU under-utilization and/or excessive communication overhead. On the other hand, for the optimistic paradigm we may even have rollback thrashing effects, with a consequent reduction of the percentage of productive (ie not rolled back) work carried out. This paper presents the design of a global memory management architecture supporting application-transparent migration of simulation objects whose state is scattered across dynamically allocated memory chunks. Our approach is based on a non-intrusive background protocol that provides each instance of the simulation kernel with information on the current mapping of the virtual address space of all the other instances. Dynamic memory requests by the application layer are then locally mapped onto virtual-address ranges that maximize the likelihood of being portable onto the address space of a remote kernel instance. In this way, independently of the load-balancing trigger (or policy), we maximize the likelihood that a desirable migration across a specific couple of kernels can actually take place due to compliance of the corresponding source/destination address spaces. We have integrated the global memory manager within the ROme OpTimistic Simulator (ROOT-Sim), namely a run-time environment based on the optimistic synchronization paradigm which automatically and transparently parallelizes the execution of event-handler-based simulation programs conforming to ANSI-C. Further, we provide a contribution in the direction of widening load-balancing schemes for optimistic simulation systems by defining migration triggers and selection policies for the objects to be migrated on the basis of memory usage patterns. An experimental assessment of the architecture and of memory-oriented load balancing is also provided.

CUDA Memory Optimizations for Large Data-Structures in the Gravit Simulator

Modern GPUs open a completely new field to optimize embarrassingly parallel algorithms. Implementing an algorithm on a GPU confronts the programmer with a new set of challenges for program optimization. Especially tuning the program for the GPU memory hierarchy whose organization and performance implications are radically different from those of general purpose CPUs; and optimizing programs at the instruction-level for the GPU. In this paper we analyze different approaches for optimizing the memory usage and access patterns for GPUs and propose a class of memory layout optimizations that can take full advantage of the unique memory hierarchy of NVIDIA CUDA. Furthermore, we analyze some classical optimization techniques and how they effect the performance on a GPU. We used the Gravit gravity simulator to demonstrate these optimizations. The final optimized GPU version achieves a 87× speedup compared to the original CPU version. Almost 30% of this speedup are direct results of the optimizations discussed in this paper.

Color and texture analysis using emerging parallel architectures

While image texture is effective for use in pattern-recognition and image-analysis algorithms, textural features are time-consuming to calculate on standard CPUs. Therefore, we present novel implementations of textural-feature algorithms on graphics processors (GPUs), enabling fast color and texture analysis. Since different textural-feature calculations exhibit diverse characteristics, we focus on using general and algorithm-specific techniques to exploit the inherent parallelism and computational power of a GPU. Common operations required during the textural-feature pipeline range from streaming computations to recursive procedures, from arithmetically intensive transcendental functions to matrix operations. Some of these kernels are well-suited to GPUs, while others require considerable programming effort to fully exploit the memory hierarchy due to their memory-usage patterns. In this paper, different strategies for computing textural features on GPUs are compared with counterpart implementations on multicore CPUs, and experimental results show GPU results reaching a speedup of 500 times for certain operations.

Analysing CMS software performance using IgProf, OProfile and callgrind

The CMS experiment at LHC has a very large body of software of its own and uses extensively software from outside the experiment. Understanding the performance of such a complex system is a very challenging task, not the least because there are extremely few developer tools capable of profiling software systems of this scale, or producing useful reports.CMS has mainly used IgProf, valgrind, callgrind and OProfile for analysing the performance and memory usage patterns of our software. We describe the challenges, at times rather extreme ones, faced as we've analysed the performance of our software and how we've developed an understanding of the performance features. We outline the key lessons learnt so far and the actions taken to make improvements. We describe why an in-house general profiler tool still ends up besting a number of renowned open-source tools, and the improvements we've made to it in the recent year.