Parallel Execution Environment Research Articles

Energy and performance improvements in stencil computations on multi-node HPC systems with different network and communication topologies

Energy and performance improvements in stencil computations are relevant for both application developers and data center administrators. They appear as the fundamental scheme in many large-scale scientific simulations and workloads. Many research efforts have focused on some estimation techniques of the energy usage of HPC systems based on specific characteristics of parallel applications. In case of stencils, we have previously concentrated on detailed estimations of energy consumption and the energy-aware distribution of stencil computations on heterogeneous processors. However, we have restricted our comprehensive studies to a single heterogeneous computing node only. In this paper, we show how scheduling and optimization techniques can be applied for energy and performance improvements of stencil computations on multi-node HPC systems using different network topologies. We formulate a scheduling model together with a new Tabu Search algorithm, called Task Movement (TM), taking into account the communication hierarchies, to minimize the overall energy usage and the execution time of stencil computations. Experimental studies show that this algorithm solves the considered problem more efficiently comparing to other, simpler heuristics. We present computational experiments for a reference 7 point stencil computation pattern on three commonly used low-diameter network topologies: Fat-tree, Dragonfly, and Torus. According to our studies, the most promising multi-node HPC architecture for stencil computations is based on the Torus network concept. Finally, we argue that the proposed scheduling model and TM algorithm can be easily adopted within existing high-level parallel execution environments for stencils automatic performance tuning.

Overhead of using spare nodes

With the increasing fault rate on high-end supercomputers, the topic of fault tolerance has been gathering attention. To cope with this situation, various fault-tolerance techniques are under investigation; these include user-level, algorithm-based fault-tolerance techniques and parallel execution environments that enable jobs to continue following node failure. Even with these techniques, some programs with static load balancing, such as stencil computation, may underperform after a failure recovery. Even when spare nodes are present, they are not always substituted for failed nodes in an effective way. This article considers the questions of how spare nodes should be allocated, how to substitute them for faulty nodes, and how much the communication performance is affected by such a substitution. The third question stems from the modification of the rank mapping by node substitutions, which can incur additional message collisions. In a stencil computation, rank mapping is done in a straightforward way on a Cartesian network without incurring any message collisions. However, once a substitution has occurred, the optimal node-rank mapping may be destroyed. Therefore, these questions must be answered in a way that minimizes the degradation of communication performance. In this article, several spare node allocation and failed node substitution methods will be proposed, analyzed, and compared in terms of communication performance following the substitution. The proposed substitution methods are named sliding methods. The sliding methods are analyzed by using our developed simulation program and evaluated by using the K computer, Blue Gene/Q (BG/Q), and TSUBAME 2.5. It will be shown that when failures occur, the stencil communication performance on the K and BG/Q can be slowed around 10 times depending on the number of node failures. The barrier performance on the K can be cut in half. On BG/Q, barrier performance can be slowed by a factor of 10. Further, it will also be shown that almost no such communication performance degradation can be seen on TSUBAME 2.5. This is because TSUBAME 2.5 has an Infiniband network connected with a FatTree topology, while the K computer and BG/Q have dedicated Cartesian networks. Thus, the communication performance degradation depends on network characteristics.

Analyzing point-to-point DDS communication over desktop virtualization software

Virtualization technologies introduce additional uncertainty and overhead for distributed applications, that may challenge their timeliness. The additional software abstraction layers of the virtualization software offer powerful parallel execution environments but, at the same time, reduce the effective performance as additional software layers are introduced. This requires some fine tuning of, among others, the communication middleware software. The different resource management mechanisms of the middleware may collide with the specific algorithms replicated by the underlying virtualization software. The present work characterises the set of steps of a publish-subscribe communication middleware in a distributed system, enhancing it to suit the communication scheme of virtualized remote nodes. The potential communication steps are identified, and the overhead introduced by the execution over the virtualization software is provided by means of a data communication rate metric; a set of benchmark tests are presented that empirically evaluate the overhead and stability of the most widely used publish-subscribe (P/S) middleware named DDS (Data Distribution System for real-time systems). A general purpose cloud computing virtual machine monitor is utilized to identify the side effects and the drawbacks of the general virtualization technology. Results are obtained on a setting with two different DDS implementations over a general purpose virtualization software, together with a discussion about the potential drawbacks of the main communication operations.

FASA: A software architecture and runtime framework for flexible distributed automation systems

Abstract Modern automation systems have to cope with large amounts of sensor data to be processed, stricter security requirements, heterogeneous hardware, and an increasing need for flexibility. The challenges for tomorrow’s automation systems need software architectures of today’s real-time controllers to evolve. This article presents FASA , a modern software architecture for next-generation automation systems. FASA provides concepts for scalable, flexible, and platform-independent real-time execution frameworks, which also provide advanced features such as software-based fault tolerance and high degrees of isolation and security. We show that FASA caters for robust execution of time-critical applications even in parallel execution environments such as multi-core processors. We present a reference implementation of FASA that controls a magnetic levitation device. This device is sensitive to any disturbance in its real-time control and thus, provides a suitable validation scenario. Our results show that FASA can sustain its advanced features even in high-speed control scenarios at 1 kHz.

A Practical and Scalable Tool to Find Overlaps between Sequences.

The evolution of the next generation sequencing technology increases the demand for efficient solutions, in terms of space and time, for several bioinformatics problems. This paper presents a practical and easy-to-implement solution for one of these problems, namely, the all-pairs suffix-prefix problem, using a compact prefix tree. The paper demonstrates an efficient construction of this time-efficient and space-economical tree data structure. The paper presents techniques for parallel implementations of the proposed solution. Experimental evaluation indicates superior results in terms of space and time over existing solutions. Results also show that the proposed technique is highly scalable in a parallel execution environment.

Authentication Service in Hadoop using One Time Pad

“Big Data” - voluminous and variety of data from different sources, which demands innovative processing and analysis for decision - making analysis. The data can be either in form of structured or unstructured data. Processing big data with the traditional processing tools and the present relational database management systems tends to be a difficult task. Parallel execution environment, like Hadoop is needed for processing voluminous data. For processing the data in a open framework like Hadoop we need a highly secure authentication system for restricting the access to the confidential business data that are processed. In this paper, a novel and a simple authentication model using one time pad algorithm that removes the communication of passwords between the servers is proposed. This model tends to enhance the security in Hadoop environment.

On-line Trace Based Automatic Parallelization of Java Programs on Multicore Platforms

We propose two new approaches that automatically parallelize Java programs at runtime. These approaches, which rely on run-time trace information collected during program execution, dynamically recompile Java byte code that can be executed in parallel. One approach utilizes trace information to improve traditional loop parallelization, and the other parallelizes traces instead of loop iterations. We also describe a cost/benefit model that makes intelligent parallelization decisions, as well as a parallel execution environment to execute parallelized programs. These techniques are based on Jikes RVM. Our approach is evaluated by parallelizing sequential Java programs, and its performance is compared to that of the manually parallelized code. According to the experimental results, our approach has low overheads and achieves competitive speedups compared to the manually parallelizing code. Moreover, trace parallelization can exploit parallelism beyond loop iterations.

Parcae

Workload, platform, and available resources constitute a parallel program's execution environment. Most parallelization efforts statically target an anticipated range of environments, but performance generally degrades outside that range. Existing approaches address this problem with dynamic tuning but do not optimize a multiprogrammed system holistically. Further, they either require manual programming effort or are limited to array-based data-parallel programs. This paper presents Parcae, a generally applicable automatic system for platform-wide dynamic tuning. Parcae includes (i) the Nona compiler, which creates flexible parallel programs whose tasks can be efficiently reconfigured during execution; (ii) the Decima monitor, which measures resource availability and system performance to detect change in the environment; and (iii) the Morta executor, which cuts short the life of executing tasks, replacing them with other functionally equivalent tasks better suited to the current environment. Parallel programs made flexible by Parcae outperform original parallel implementations in many interesting scenarios.

Scalable SIMD-parallel memory allocation for many-core machines

Dynamic memory allocation is an important feature of modern programming systems. However, the cost of memory allocation in massively parallel execution environments such as CUDA has been too high for many types of kernels. This paper presents XMalloc, a high-throughput memory allocation mechanism that dramatically magnifies the allocation throughput of an underlying memory allocator. XMalloc embodies two key techniques: allocation coalescing and buffering using efficient queues. This paper describes these two techniques and presents our implementation of XMalloc as a memory allocator library. The library is designed to be called from kernels executed by massive numbers of threads. Our experimental results based on the NVIDIA G480 GPU show that XMalloc magnifies the allocation throughput of the underlying memory allocator by a factor of 48.

Diffusion tensor fiber tracking on graphics processing units

Diffusion tensor magnetic resonance imaging has been successfully applied to the process of fiber tracking, which determines the location of fiber bundles within the human brain. This process, however, can be quite lengthy when run on a regular workstation. We present a means of executing this process by making use of the graphics processing units of computers’ video cards, which provide a low-cost parallel execution environment that algorithms like fiber tracking can benefit from. With this method we have achieved performance gains varying from 14 to 40 times on common computers. Because of accuracy issues inherent to current graphics processing units, we define a variation index in order to assess how close the results obtained with our method are to those generated by programs running on the central processing units of computers. This index shows that results produced by our method are acceptable when compared to those of traditional programs.

Design for AJACS, yet another Java constraint programming framework

This article introduces AJACS (Another Java Constraint Programming System), a toolkit for Concurrent Constraint programming implemented in the Java language. It comes as a successor to our previous work in implementing Constraint Programming idioms in Java, GC [5], in that it represents an attempt to deal with some of GC's inadequacies in terms of performance whilst providing a setting which is adequate to express problems in a way that can be easily solved in a parallel execution environment, as provided by a concurrent programming setting.We claim that AJACS provides a very flexible toolkit for use in general applications that may benefit from constraint programming techniques. We also claim that AJACS allows for the coding of CSP problems whose solution is amenable to a practically effortless parallelization.

Experiences with the SEMPA Resource Manager for Real World Scientific

The Sepma Resourse Manager is a resource management system for parallel applications in a network of workstations that enables the utilization of idle resources for parallel computing and shortens the execution time of parallel applications by dynamic load balancing at runtime. The network of workstations runs in a non-dedicated mode, where the Sepma Resourse Manager assigns hosts that are not required by interactive users to its batch jobs. The flexible structure of the Sepma Resourse Manager guarantees scalability and results in functions for transparent process migration and dynamic reconfiguration of the parallel execution environment. We present performance measurements with an industrial computational fluid dynamics program proving the efficiency of the Sepma Resourse Manager.

Extensible Parallel Query Processing for Exploratory Geoscientific Data Mining

Exploratory data mining and analysis requires a computing environment which provides facilities for the user-friendly expression and rapid execution of “scientific queries.” In this paper, we address research issues in the parallelization of scientific queries containing complex user-defined operations. In a parallel query execution environment, parallelizing a query execution plan involves determining how input data streams to evaluators implementing logical operations can be divided to be processed by clones of the same evaluator in parallel. We introduced the concept of “relevance window” that characterizes data lineage and data partitioning opportunities available for an user-defined evaluator. In addition, we developed a query parallelization framework by extending relational parallel query optimization algorithms to allow the parallelization characteristics of user-defined evaluators to guide the process of query parallelization in an extensible query processing environment. We demonstrated the utility of our system by performing experiments mining cyclonic activity, blocking events, and the upward wave-energy propagation features from several observational and model simulation datasets.

JESSICA: Java-Enabled Single-System-Image Computing Architecture

JESSICA stands for Java-enabled single-system-image computing architecture, a middleware that runs on top of the standard UNIX operating system to support parallel execution of multithreaded Java applications in a cluster of computers. JESSICA hides the physical boundaries between machines and makes the cluster appear as a single computer to applications—a single system image. JESSICA supports preemptive thread migration, which allows a thread to freely move between machines during its execution, and global object sharing through the help of a distributed shared-memory subsystem. JESSICA implements location-transparency through a message-redirection mechanism. The result is a parallel execution environment where threads are automatically redistributed across the cluster for achieving the maximal possible parallelism. A JESSICA prototype that runs on a Linux cluster has been implemented and considerable speedups have been obtained for all the experimental applications tested.

Finding, expressing and managing parallelism in programs executed on clusters of workstations

The goal of this paper to identify and discuss the basic issues of and solutions to parallel processing on clusters of workstations (COWs). Firstly, identification and expressing parallelism in application programs are discussed. The following approaches to finding and expressing parallelism are characterized: parallel programming languages, parallel programming tools, sequential programming supported by distributed shared memory (DSM), and parallelising compilers. Secondly, efficient management of available parallelism is discussed. As parallel execution requires an efficient management of processes and computational resources, a parallel execution environment proposed here is to be built based on a distributed operating system. This system, in order to allow parallel programs to achieve high performance and transparency, should provide services such as global scheduling, process migration, local and remote process creation, computation coordination, group communication and distributed shared memory.

The CAPDYN environment and its message-passing library implementation

We present here the implementation of a parallel programming and execution environment for the CAPITAN computer on top of the basic functionalities of the native environment. We explain how we implemented some OS level functionalities as well as the NX, PVM and MPI interface programming interfaces. Then we survey some of the issues to achieve low-latency communications at the user-level.

The object-oriented development of a parallel application in protein dynamics: why we need software tools for HPCN applications

We analyse the concurrency and performance of the various types of force calculations involved in the molecular dynamics simulation of large protein or polyelectrolyte molecules. Although this analysis can in principle be used to write a meta-program to optimize load-balancing of this application on an MPP system, we argue that it is an enormous undertaking not appropriate for the computational scientist. Instead we argue that it is better to exploit research in parallel execution environments which provide automatic load-balancing for concurrent Object-Oriented applications. We also argue that use of Object-Oriented technology in the design of simulation software encapsulates the natural concurrency of the system. We illustrate this point with a discussion of the constraint force calculation for a polymeric molecule.

Performance degradation due to multiprogramming and system overheads in real workloads

In this paper, performance degradation specifically due to the multiprogramming (MP) overhead in a parallel execution environment is quantified. In addition, total system overhead is also measured. A methodology, which estimates the MP overhead present in real workloads, is illustrated with real measurements taken on an Alliant FX/80 running Xylem (Cedar's operating system). It is found that MP overhead usually consumes between 10% and 23% of the processing power available to parallel programs. Total system overhead usually consumes between 12% and 30% of the parallel environment processing power, but is found to be as high as 82.1%. The mean MP overhead is determined to be 16% which is well over half the total system overhead executed on the system (the mean system overhead is determined to be 24% of the processing power). It is found that MP overhead, total system overhead, and application completion time are all moderately correlated. Relationships between the characteristics of a workload and the overhead measurements indicate that processor utilization and to a lesser degree paging are moderately correlated with the overhead present in the workload. It is also found that MP overhead is statistically independent of the number of parallel jobs in the system, while total system overhead is not.