High Performance Parallel Systems Research Articles

A Study of Failure Recovery and Logging of High-Performance Parallel File Systems

Large-scale parallel file systems (PFSs) play an essential role in high-performance computing (HPC). However, despite their importance, their reliability is much less studied or understood compared with that of local storage systems or cloud storage systems. Recent failure incidents at real HPC centers have exposed the latent defects in PFS clusters as well as the urgent need for a systematic analysis. To address the challenge, we perform a study of the failure recovery and logging mechanisms of PFSs in this article. First, to trigger the failure recovery and logging operations of the target PFS, we introduce a black-box fault injection tool called PFault , which is transparent to PFSs and easy to deploy in practice. PFault emulates the failure state of individual storage nodes in the PFS based on a set of pre-defined fault models and enables examining the PFS behavior under fault systematically. Next, we apply PFault to study two widely used PFSs: Lustre and BeeGFS. Our analysis reveals the unique failure recovery and logging patterns of the target PFSs and identifies multiple cases where the PFSs are imperfect in terms of failure handling. For example, Lustre includes a recovery component called LFSCK to detect and fix PFS-level inconsistencies, but we find that LFSCK itself may hang or trigger kernel panics when scanning a corrupted Lustre. Even after the recovery attempt of LFSCK, the subsequent workloads applied to Lustre may still behave abnormally (e.g., hang or report I/O errors). Similar issues have also been observed in BeeGFS and its recovery component BeeGFS-FSCK. We analyze the root causes of the abnormal symptoms observed in depth, which has led to a new patch set to be merged into the coming Lustre release. In addition, we characterize the extensive logs generated in the experiments in detail and identify the unique patterns and limitations of PFSs in terms of failure logging. We hope this study and the resulting tool and dataset can facilitate follow-up research in the communities and help improve PFSs for reliable high-performance computing.

ARENA: Asynchronous Reconfigurable Accelerator Ring to Enable Data-Centric Parallel Computing

The next generation HPC and data centers are likely to be reconfigurable and data-centric due to the trend of hardware specialization and the emergence of data-driven applications. In this article, we propose ARENA - an asynchronous reconfigurable accelerator ring architecture as a potential scenario on how the future HPC and data centers will be like. Despite using the coarse-grained reconfigurable arrays (CGRAs) as the substrate platform, our key contribution is not only the CGRA-cluster design itself, but also the ensemble of a new architecture and programming model that enables asynchronous tasking across a cluster of reconfigurable nodes, so as to bring specialized computation to the data rather than the reverse. We presume distributed data storage without asserting any prior knowledge on the data distribution. Hardware specialization occurs at runtime when a task finds the majority of data it requires are available at the present node. In other words, we dynamically generate specialized CGRA accelerators where the data reside. The asynchronous tasking for bringing computation to data is achieved by circulating the task token, which describes the dataflow graphs to be executed for a task, among the CGRA cluster connected by a fast ring network. Evaluations on a set of HPC and data-driven applications across different domains show that ARENA can provide better parallel scalability with reduced data movement (53.9 percent). Compared with contemporary compute-centric parallel models, ARENA can bring on average 4.37× speedup. The synthesized CGRAs and their task-dispatchers only occupy 2.93mm$^2$2<; inline-graphic xlink:href="tan-ieq1-3081074.gif"/> chip area under 45nm process technology and can run at 800MHz with on average 759.8mW power consumption. ARENA also supports the concurrent execution of multi-applications, offering ideal architectural support for future high-performance parallel computing and data analytics systems.

Leveraging InfiniBand controller to configure deadlock-free routing engines for Dragonflies

The Dragonfly topology is currently one of the most popular network topologies in high-performance parallel systems. The interconnection networks of many of these systems are built from components based on the InfiniBand specification. However, due to some constraints in this specification, the available versions of the InfiniBand network controller (OpenSM) do not include routing engines based on some popular deadlock-free routing algorithms proposed theoretically for Dragonflies, such as the one proposed by Kim and Dally based on Virtual-Channel shifting. In this paper we propose a straightforward method to integrate this routing algorithm in OpenSM as a routing engine, explaining in detail the configuration required to support it. We also provide experiment results, obtained both from a real InfiniBand-based cluster and from simulation, to validate the new routing engine and to compare its performance and requirements against other routing engines currently available in OpenSM.

An offline framework for high-dimensional ensemble Kalman filters to reduce the time to solution

Abstract. The high computational resources and the time-consuming IO (input/output) are major issues in offline ensemble-based high-dimensional data assimilation systems. Bearing these in mind, this study proposes a sophisticated dynamically running job scheme as well as an innovative parallel IO algorithm to reduce the time to solution of an offline framework for high-dimensional ensemble Kalman filters. The dynamically running job scheme runs as many tasks as possible within a single job to reduce the queuing time and minimize the overhead of starting and/or ending a job. The parallel IO algorithm reads or writes non-overlapping segments of multiple files with an identical structure to reduce the IO times by minimizing the IO competitions and maximizing the overlapping of the MPI (Message Passing Interface) communications with the IO operations. Results based on sensitive experiments show that the proposed parallel IO algorithm can significantly reduce the IO times and have a very good scalability, too. Based on these two advanced techniques, the offline and online modes of ensemble Kalman filters are built based on PDAF (Parallel Data Assimilation Framework) to comprehensively assess their efficiencies. It can be seen from the comparisons between the offline and online modes that the IO time only accounts for a small fraction of the total time with the proposed parallel IO algorithm. The queuing time might be less than the running time in a low-loaded supercomputer such as in an operational context, but the offline mode can be nearly as fast as, if not faster than, the online mode in terms of time to solution. However, the queuing time is dominant and several times larger than the running time in a high-loaded supercomputer. Thus, the offline mode is substantially faster than the online mode in terms of time to solution, especially for large-scale assimilation problems. From this point of view, results suggest that an offline ensemble Kalman filter with an efficient implementation and a high-performance parallel file system should be preferred over its online counterpart for intermittent data assimilation in many situations.

A Machine Learning-based Algorithm for Water Network Contamination Source Localization.

In this paper, a novel machine learning based algorithm for water supply pollution source identification is presented built specifically for high performance parallel systems. The algorithm utilizes the combination of Artificial Neural Networks for classification of the pollution source with Random Forests for regression analysis to determine significant variables of a contamination event such as start time, end time and contaminant chemical concentration. The algorithm is based on performing Monte Carlo water quality and hydraulic simulations in parallel, recording data with sensors placed within a water supply network and selecting a most probable pollution source based on a tournament style selection between suspect nodes in a network with mentioned machine learning methods. The novel algorithmic framework is tested on a small (92 nodes) and medium sized (865 nodes) water supply sensor network benchmarks with a set contamination event start time, end time and chemical concentration. Out of the 30 runs, the true source node was the finalist of the algorithm’s tournament style selection for 30/30 runs for the small network, and 29/30 runs for the medium sized network. For all the 30 runs on the small sensor network, the true contamination event scenario start time, end time and chemical concentration was set as 14:20, 20:20 and 813.7 mg/L, respectively. The root mean square errors for all 30 algorithm runs for the three variables were 48 min, 4.38 min and 18.06 mg/L. For the 29 successful medium sized network runs the start time was 06:50, end time 07:40 and chemical concentration of 837 mg/L and the root mean square errors were 6.06 min, 12.36 min and 299.84 mg/L. The algorithmic framework successfully narrows down the potential sources of contamination leading to a pollution source identification, start and ending time of the event and the contaminant chemical concentration.

A Comprehensive Study of MapReduce Over Lustre for Intermediate Data Placement and Shuffle Strategies on HPC Clusters

With high performance interconnects and parallel file systems, running MapReduce over modern High Performance Computing (HPC) clusters has attracted much attention due to its uniqueness of solving data analytics problems with a combination of Big Data and HPC technologies. Since the MapReduce architecture relies heavily on the availability of local storage media, the Lustre-based global storage in HPC clusters poses many new opportunities and challenges. In this paper, we perform a comprehensive study on different MapReduce over Lustre deployments and propose a novel high-performance design of YARN MapReduce on HPC clusters by utilizing Lustre as the additional storage provider for intermediate data. With a deployment architecture where both local disks and Lustre are utilized for intermediate data storage, we propose a novel priority directory selection scheme through which RDMA-enhanced MapReduce can choose the best intermediate storage during runtime by on-line profiling. Our results indicate that, we can achieve 44 percent performance benefit for shuffle-intensive workloads in leadership-class HPC systems. Our priority directory selection scheme can improve the job execution time by 63 percent over default MapReduce while executing multiple concurrent jobs. To the best of our knowledge, this is the first such comprehensive study for YARN MapReduce with Lustre and RDMA.

Assessing the Suitability of King Topologies for Interconnection Networks

In the late years many different interconnection networks have been used with two main tendencies. One is characterized by the use of high-degree routers with long wires while the other uses routers of much smaller degree. The latter rely on two-dimensional mesh and torus topologies with shorter local links. This paper focuses on doubling the degree of common 2D meshes and tori while still preserving an attractive layout for VLSI design. By adding a set of diagonal links in one direction, diagonal networks are obtained. By adding a second set of links, networks of degree eight are built, named king networks . This research presents a comprehensive study of these networks which includes a topological analysis, the proposal of appropriate routing procedures and an empirical evaluation. King networks exhibit a number of attractive characteristics which translate to reduced execution times of parallel applications. For example, the execution times NPB suite are reduced up to a 30 percent. In addition, this work reveals other properties of king networks such as perfect partitioning that deserves further attention for its convenient exploitation in forthcoming high-performance parallel systems.

Performance evaluation of SD – cards by "system-on-chip" technology

Subject matter. We propose the hardware and software block for performing write or read operation and testing the performance of different SD-cards using the technology of "system-on-chip" in integrated environment by Altera. This task is relevant for the design of high-performance parallel systems on a chip, where the lower limit for writing speed (and/or reading) is critical and significant. Method. The proposed approach is based on three items. First, schemes are used with programmable logic to create a compact hardware and software unit as part of video system on chip. Secondly, reprogrammable structures are applied that support serial and parallel modes of exchange. Thirdly, the increment of data bus width is possible, via the serial transfer and the parallel connection of multiple units simultaneously. Main Results. The analysis of test results of different memory cards has shown that the speed of writing and reading for all cards differs from the declared one by manufacturer, and the error rate of writing (and reading) of data is 30...50% (for reading ~30%). A maximum download speed of memory cards is possible by increasing the number of cascade-connected blocks. The results show that the simplest implementation of this hardware-software unit requires resources of the order of 2000-2500 elementary logic cells. Each cell implements the logical operation "AND-OR with memory", while the resource ЕР3С25F256C8 of the popular chip family Cyclone III consists of 5976 (26%) of these cells with the chip price equal to $5-10 apiece. Practical Relevance. The authors believe that the proposed hardware-software unit can be regarded as the cheapest specialized component in the ECB domestic microelectronics industry to create video information in real time systems.

A Practical Data Classification Framework for Scalable and High Performance Chip-Multiprocessors

State-of-the-art chip multiprocessor (CMP) proposals emphasize general optimizations designed to deliver computing power for many types of applications. Potentially, significant performance improvements that leverage application-specific characteristics such as data access behavior are missed by this approach. In this paper, we demonstrate how scalable and high-performance parallel systems can be built by classifying data accesses into different categories and treating them differently. We develop a novel compiler-based approach to speculatively detect a data classification termed practically private, which we demonstrate is ubiquitous in a wide range of parallel applications. Leveraging this classification provides efficient solutions to mitigate data access latency and coherence overhead in today’s many-core architectures. While the proposed data classification scheme can be applied to many micro-architectural constructs including the TLB, coherence directory, and interconnect, we demonstrate its potential through an efficient cache coherence design. Specifically, we show that the compiler-assisted mechanism reduces an average of 46% coherence traffic and achieves up to 12%, 8%, and 5% performance improvement over shared, private, and state-of-the-art NUCA-based caching, respectively, depending on scenarios.

Implementation and Evaluation of mpiBLAST-PIO on HPC Cluster

to exponential growth in the size of genomic databases, traditional techniques of sequence search proved to be slow. To address the above problem, an open source and parallel version of BLAST called mpiBLAST was developed by the programmers. In mpiBLAST, the master process distributes the database fragments among worker nodes to compute the sequence search in parallel. As merging and writing of the results is done sequentially by the master process, it would create performance bottleneck with increasing number of processors and varying database sizes. To handle this high non-search overhead, mpiBLAST-PIO was introduced. This paper describes the optimized and extended version of mpiBLAST called mpiBLAST-PIO. The goal of this research was to investigate the performance of parallel implementation of BLAST in comparison to sequential NCBI-BLAST by measuring Speedup and efficiency on HPC platform using Infiniband. Different options of mpiBLAST-PIO were activated that helped in understanding the optimal parameters for achieving highly scalable parallel BLAST implementation. The results found that parallel-writing of the results, can evolve as an efficient solution when high-performance parallel file system is available.

Xrootd data access for LHC experiments at the INFN-CNAF Tier-1

The Mass Storage System installed at the INFN-CNAF Tier-1 is one of the biggest hierarchical storage facilities in Europe. It currently provides storage resources for about 12% of all LHC data, as well as for other experiments. The Grid Enabled Mass Storage System (GEMSS) is the current solution implemented at CNAF and it is based on a custom integration between a high performance parallel file system (General Parallel File System, GPFS) and a tape management system for long-term storage on magnetic media (Tivoli Storage Manager, TSM). Data access to Grid users is being granted since several years by the Storage Resource Manager (StoRM), an implementation of the standard SRM interface, widely adopted within the WLCG community.The evolving requirements from the LHC experiments and other users are leading to the adoption of more flexible methods for accessing the storage. These include the implementation of the so-called storage federations, i.e. geographically distributed federations allowing direct file access to the federated storage between sites.A specific integration between GEMSS and Xrootd has been developed at CNAF to match the requirements of the CMS experiment. This was already implemented for the ALICE use case, using ad-hoc Xrootd modifications. The new developments for CMS have been validated and are already available in the official Xrootd builds. This integration is currently in production and appropriate large scale tests have been made.In this paper we present the Xrootd solutions adopted for ALICE, CMS, ATLAS and LHCb to increase the availability and optimize the overall performance.

New Developments in FEFF: FEFF9 and JFEFF

The ab initio core-level spectroscopy code FEFF9 has seen many new developments in recent years. We describe the addition of new physics and new features designed to calculate more accurate spectra. We also present the user-friendly Java-based GUI JFEFF that simplifies running FEFF on platforms ranging from personal computers to high-performance parallel systems and virtual cloud platforms.

SPH Based Fluid Animation Using CUDA Enabled GPU

Realistic Fluid Animation is an inherent part of special effects in Film and Gaming Industry . These animations are created through the simulation of highly compute intensive fluid model . The computations involved in execution of fluid modelemphasize the need of high performance parallel system to achieve the real time animation. This paper primarily devoted to the formalization of parallel algorithms for fluid animation employing Smoothed Particle Hydrodynamics (SPH) model onCompute Unified Device Architecture (CUDA). We have demonstrated a considerable execution speedu p on CUDA as compare to CPU. The speedup is further improved by reducingcomplexity of SPH computations from O(N 2 ) to O(N) by utilizing spatial grid based particle neighbou r lookup.

Editorial: enabling technologies for programming extreme scale systems

Multi-core architecture presents a new trend, and core-based parallel processing algorithms will continue to become more important. In addition, Grid and Cloud Computing (GCC) has emerged rapidly as an exciting new paradigm that offers a challenging model of computing and poses fascinating problems ranging from parallel architecture to programming models and compiler optimization. Furthermore, the wide-scale deployment of computing nodes, clusters, and embedded processors will continue to fuel core parallel processing and distributed systems algorithms. Thus, the importance of scalable parallel programming will continue to grow. The field of parallel and distributed computing is going through rapid changes. The emerging multi-core commodity processors and advanced networking technologies are allowing for the design of cost-effective parallel and distributed systems. To achieve scalability and high performance in these systems, it is increasingly becoming challenging to design better architecture, software environments, networking protocols, compilers, operating systems, languages, algorithms and applications for designing next generation scalable and high-performance parallel systems. This special issue is intended to foster stateof-the-art research in the area of extreme scale parallel systems including the topics of parallel algorithm design, data flow analysis, virtualization, energy efficient, load balancing, parallel computing model, multi-core programming, GPU implementation and optimization, execution on real-world parallel architecture and novel applications associated with this new paradigm. The papers included in this special issue demonstrate the effectiveness and efficiency of a variety of technologies and applications in parallel, grid, cloud and extreme scale systems. All of them not only contribute valuable insights and results but

Orchestrating computational algebra components into a high-performance parallel system

This paper demonstrates that it is possible to obtain good, scalable parallel performance by coordinating multiple instances of unaltered sequential computational algebra systems in order to deliver a single parallel system. The paper presents the first substantial parallel performance results for SymGrid-Par, a system that orchestrates computational algebra components into a high-performance parallel application. We show that SymGrid-Par is capable of exploiting different parallel/multicore architectures without any change to the computational algebra component. Ultimately, our intention is to extend our system so that it is capable of orchestrating heterogeneous computations across a high-performance computational grid.

Gateway: New high performance computing facility for EFDA task force on integrated Tokamak modelling

The EFDA task force on Integrated Tokamak Modelling (ITM-TF) is providing the EU fusion community with a complete and flexible suite of reliable software tools and codes able to simulate the next ITER and DEMO plasma discharges. EFDA has launched the Gateway project as the first computing facility to be jointly used by EU fusion associations. The Gateway has been designed to allow the ITM-TF members to work together on a common platform and share their codes, developments tools and technologies as well as to make able the inter-operation with tera-scale supercomputer facilities. Technically the Gateway is a rather small high computing facility in operation since 2008, with 1 Teraflops of theoretical peak and 100 Terabytes shared storage area for experimental and simulation data. This paper describes the information technologies involved in the Gateway facility, particularly as regards the low latency interconnect network for multi-core platform as well as fast I/O solutions implemented with storage over Infiniband and high performance parallel file systems. The solutions adopted on the Gateway facility had undergone to specific benchmarks showing excellent performances in typical fusion data handling issues. Finally the interoperability between Gateway and ENEA CRESCO supercomputer will be described showing the access solutions that allow the exploitation of tera-scale supercomputer facilities.

A concurrent dynamic analysis framework for multicore hardware

Software has spent the bounty of Moore's law by solving harder problems and exploiting abstractions, such as high-level languages, virtual machine technology, binary rewriting, and dynamic analysis. Abstractions make programmers more productive and programs more portable, but usually slow them down. Since Moore's law is now delivering multiple cores instead of faster processors, future systems must either bear a relatively higher cost for abstractions or use some cores to help tolerate abstraction costs. This paper presents the design, implementation, and evaluation of a novel concurrent, configurable dynamic analysis framework that efficiently utilizes multicore cache architectures. It introduces Cache-friendly Asymmetric Buffering (CAB), a lock-free ring-buffer that implements efficient communication between application and analysis threads. We guide the design and implementation of our framework with a model of dynamic analysis overheads. The framework implements exhaustive and sampling event processing and is analysis-neutral. We evaluate the framework with five popular and diverse analyses, and show performance improvements even for lightweight, low-overhead analyses. Efficient inter-core communication is central to high performance parallel systems and we believe the CAB design gives insight into the subtleties and difficulties of attaining it for dynamic analysis and other parallel software.

Comments on "Low Diameter Interconnections for Routing in High-Performance Parallel Systems," with Connections and Extensions to Arc Coloring of Coset Graphs

For original paper see R. Melhem, ibid., vol.56, p.502-10, (2007). Recently, Melhem presented a new class of low-diameter interconnection (LDI) networks, (IEEE Trans. computers, Vol. 56, No. 4, pp. 502-510). We note that LDI networks are the same as the previously known generalized de Bruijn graphs, point out an error in the decomposition of LDI networks into permutations, and find that the correct decomposition scheme is an instance of arc coloring for coset graphs. Hence, we pursue a number of general results on arc coloring of coset graphs that can be applied to this particular decomposition problem as well as within many other contexts, including complete arc coloring and normality of coset graphs.

BioGraphE: high-performance bionetwork analysis using the Biological Graph Environment

BackgroundGraphs and networks are common analysis representations for biological systems. Many traditional graph algorithms such as k-clique, k-coloring, and subgraph matching have great potential as analysis techniques for newly available data in biology. Yet, as the amount of genomic and bionetwork information rapidly grows, scientists need advanced new computational strategies and tools for dealing with the complexities of the bionetwork analysis and the volume of the data.ResultsWe introduce a computational framework for graph analysis called the Biological Graph Environment (BioGraphE), which provides a general, scalable integration platform for connecting graph problems in biology to optimized computational solvers and high-performance systems. This framework enables biology researchers and computational scientists to identify and deploy network analysis applications and to easily connect them to efficient and powerful computational software and hardware that are specifically designed and tuned to solve complex graph problems. In our particular application of BioGraphE to support network analysis in genome biology, we investigate the use of a Boolean satisfiability solver known as Survey Propagation as a core computational solver executing on standard high-performance parallel systems, as well as multi-threaded architectures.ConclusionIn our application of BioGraphE to conduct bionetwork analysis of homology networks, we found that BioGraphE and a custom, parallel implementation of the Survey Propagation SAT solver were capable of solving very large bionetwork problems at high rates of execution on different high-performance computing platforms.

Low Diameter Interconnections for Routing in High-Performance Parallel Systems

A new class of low diameter interconnections (LDI) is proposed for high-performance computer systems that are augmented with circuit switching networks. In these systems, the network is configured to match the communication patterns of applications, when these patterns exhibit temporal locality, and to embed a logical topology to route traffic that does not exhibit locality. The new LDI topology is a surprisingly simple directed graph which minimizes the network diameter for a given node degree and number of nodes. It can be easily embedded in circuit switching networks to route random traffic with high bandwidth and low latency