Network Of Workstations Research Articles

Event Processing Performance of KIAF Based on PROOF

Managing and processing the large scale of data is the crucial part of the experimental high energy physics which uses a gigantic instrument in order to exploit the world’s largest particle accelerator at CERN. The WLCG has enabled the distributed computing technology, which is called the grid, and has proven its excellent performance by contributing their capability of data management and processing. In addition to the grid computing, a compact but powerful computing facility introducing the parallelism is designed. In this paper, we introduce the KIAF cluster which is designed to process large scale of data produced from the ALICE experiment at CERN LHC in parallel based on PROOF. The PROOF enables parallelism on a Linux cluster by exploiting the special characteristic of the data produced in high energy physics. Event processing performance, the number of event processed per second and the size of data processed per second, of the KIAF cluster is shown with the practical use case for the high energy physics. The performance of the KIAF shows pseudo-linearity increasing as the number of workers involved in the processing while it shows as well an evidence of the upper limitation of the scalability of PROOF cluster in terms of the performance.

The ATLAS ARC backend to HPC

The current distributed computing resources used for simulating and processing collision data collected by ATLAS and the other LHC experiments are largely based on dedicated x86 Linux clusters. Access to resources, job control and software provisioning mechanisms are quite different from the common concept of self-contained HPC applications run by particular users on specific HPC systems. We report on the development and the usage in ATLAS of a SSH backend to the Advanced Resource Connector (ARC) middleware to enable HPC compliant access and on the corresponding software provisioning mechanisms.

A parallel hybrid metaheuristic for bicluster editing

The bicluster editing problem (BEP) consists of editing (adding or removing) the edges of a bipartite graph G in order to transform it into a vertex-disjoint union of complete bipartite subgraphs, in such a way that the sum of the weights of the edited edges is minimum. In this paper, we propose five parallel strategies for the implementation of a hybrid metaheuristic for the BEP, consisting of a GRASP with VNS as local search. Computational experiments show near-linear speedups on Linux cluster with 64 processors and better solutions than those of the sequential approach.

Advanced 3D imaging of complex geoelectrical structures using towed streamer EM data over the Mariner field in the North Sea

Towed streamer electromagnetic (EM) data acquired at the Mariner heavy oilfield in the UK sector of the North Sea have been inverted using a fast and efficient 3D anisotropic inversion code. The towed streamer EM system was towed from a single vessel. The system consisted of a horizontal bipole source and electrode sensors housed in a streamer cable. This enabled a densely sampled grid of data over the subsurface volume of interest. The purpose was to estimate the resistivity structure in a volume including the Maureen and Heimdal reservoir structures in the Mariner complex. The 3D inversion algorithm is based on the contraction integral equation method and utilizes a re-weighted regularized conjugate gradient technique to minimize an objective functional (e.g., Zhdanov et al., 2014). A moving sensitivity domain approach is introduced to handle the large amount of data over the large area (Zhdanov, 2010; Zhdanov and Cox, 2012; Zhdanov et al., 2014; Cox and Zhdanov, 2014). This inversion method is proven to be fast and efficient and is here shown to be suitable for towed streamer EM data from complex geological environments such as the Mariner area. In this case, the final 3D resistivity cube after inversion and with a corresponding normalized misfit of 5.4% correlates well with the expected structure from seismic data and well logs. In particular, the 3D inversion was able to extract an anomaly with vertical and horizontal resistivities of 8-10 and 4-5 Ohm-m, respectively, corresponding to the Maureen and Heimdal reservoirs close to the resistive chalk and basement. The run time on a PC cluster was only seven hours for the full 3D inversion with data from all survey lines covering the area of interest.

GA‐Based Multi‐Objective Optimization for Retrofit Design on a Multi‐Core PC Cluster

AbstractThis article presents a distributed nondominated sorting genetic algorithm II (NSGA‐II) for optimal seismic retrofit design using buckling restrained braces (BRBs) on a cluster of multi‐core PCs. In the formulation, two conflicting objective functions of the initial BRB installation cost required for seismic retrofitting and damage cost that can be incurred by earthquakes expected during the life cycle of the structure were minimized. Because time‐consuming nonlinear structural analyses are required for fitness evaluations of individuals in every generation, parallelism at candidate design level or individual level is exploited by assigning fitness evaluations for individuals to slave core processors evenly. The distributed algorithm is applied to seismic retrofit design of 2D steel frame structure and 3D irregular reinforced concrete structure. The performance of the distributed NSGA‐II was assessed based on three criteria: convergence of the distributed algorithm, efficiency of distributed computing, and quality of optimal solutions. Implementation of the distributed algorithm on the multi‐core cluster consisting of up to 64 core processors resulted in relatively high speedups or efficiencies of the distributed optimization without deteriorating the quality of the optimal solutions.

Fault tolerance at system level based on RADIC architecture

The increasing failure rate in High Performance Computing encourages the investigation of fault tolerance mechanisms to guarantee the execution of an application in spite of node faults. This paper presents an automatic and scalable fault tolerant model designed to be transparent for applications and for message passing libraries. The model consists of detecting failures in the communication socket caused by a faulty node. In those cases, the affected processes are recovered in a healthy node and the connections are reestablished without losing data. The Redundant Array of Distributed Independent Controllers architecture proposes a decentralized model for all the tasks required in a fault tolerance system: protection, detection, recovery and masking. Decentralized algorithms allow the application to scale, which is a key property for current HPC system. Three different rollback recovery protocols are defined and discussed with the aim of offering alternatives to reduce overhead when multicore systems are used. A prototype has been implemented to carry out an exhaustive experimental evaluation through Master/Worker and Single Program Multiple Data execution models. Multiple workloads and an increasing number of processes have been taken into account to compare the above mentioned protocols. The executions take place in two multicore Linux clusters with different socket communications libraries.

A parallel finite element study of 3D mixed convection in a fluid saturated cubic porous enclosure under injection/suction effect

This study examines natural convection from a hot cubical structure buried inside a fluid saturated porous media under suction/injection induced forced flow conditions. The numerical computations are carried out by a segregated finite element approach under the paradigm of domain decomposition technique. Suitable sparse data structures and algorithms are designed for efficient implementation on a cluster of PCs called ANUCLUSTER under ANULIB parallel processing environment. The parallel numerical simulations are carried out for various values of Ra and for different sizes of isothermal hot cubical objects. Results have been presented in the form of Nusselt number, temperature and velocity vector plots.

A convergence of key‐value storage systems from clouds to supercomputers

SummaryThis paper presents a convergence of distributed key‐value storage systems in clouds and supercomputers. It specifically presents ZHT, a zero‐hop distributed key‐value store system, which has been tuned for the requirements of high‐end computing systems. ZHT aims to be a building block for future distributed systems, such as parallel and distributed file systems, distributed job management systems, and parallel programming systems. ZHT has some important properties, such as being lightweight, dynamically allowing nodes join and leave, fault tolerant through replication, persistent, scalable, and supporting unconventional operations such as append, compare and swap, callback in addition to the traditional insert/lookup/remove. We have evaluated ZHT's performance under a variety of systems, ranging from a Linux cluster with 64 nodes, an Amazon EC2 virtual cluster up to 96 nodes, to an IBM Blue Gene/P supercomputer with 8K nodes. We compared ZHT against other key‐value stores and found it offers superior performance for the features and portability it supports. This paper also presents several real systems that have adopted ZHT, namely, FusionFS (a distributed file system), IStore (a storage system with erasure coding), MATRIX (distributed scheduling), Slurm++ (distributed HPC job launch), Fabriq (distributed message queue management); all of these real systems have been simplified because of key‐value storage systems and have been shown to outperform other leading systems by orders of magnitude in some cases. It is important to highlight that some of these systems are rooted in HPC systems from supercomputers, while others are rooted in clouds and ad hoc distributed systems; through our work, we have shown how versatile key‐value storage systems can be in such a variety of environments. Copyright © 2015 John Wiley & Sons, Ltd.

An integrated approach to workflow mapping and task scheduling for delay minimization in distributed environments

Many scientific applications feature large-scale workflows consisting of computing modules that must be strategically deployed and executed in distributed environments. The end-to-end performance of such scientific workflows depends on both the mapping scheme that determines module assignment, and the scheduling policy that determines resource allocation if multiple modules are mapped to the same node. These two aspects of workflow optimization are traditionally treated as two separated topics, and the interactions between them have not been fully explored by any existing efforts. As the scale of scientific workflows and the complexity of network environments rapidly increase, each individual aspect of performance optimization alone can only meet with limited success. We conduct an in-depth investigation into workflow execution dynamics in distributed environments and formulate a generic problem that considers both workflow mapping and task scheduling to minimize the end-to-end delay of workflows. We propose an integrated solution, referred to as Mapping and Scheduling Interaction (MSI), to improve the workflow performance. The efficacy of MSI is illustrated by both extensive simulations and proof-of-concept experiments using real-life scientific workflows for climate modeling on a PC cluster.

High Performance Computing Clusters Design & Analysis Using Red Hat Enterprise Linux

The purpose of this paper was to configure a cluster computing system to improve performance over that of a single computer, while typically being much more cost-effective than single computers of comparable speed. High performance computing more and more issued in all kinds of fields. But, for the common user, there are several questions on implementing the process of competing on the specialized clusters: such as expense high, management difficulty and operation complex and so on. To overcome these problems, this article designs and realizes the high performance computing environment based on Linux cluster after studying the key technology of cluster and parallel computing. Finally, the performance of system environment is tested by interactive version of CPI algorithm. Before discussion on HPC the cluster computing system with its classification, the advantages and application of clustering system will also be discussed here. DOI: http://dx.doi.org/10.11591/telkomnika.v14i3.7798 Full Text: PDF

A Distributed Memory Hierarchy and Data Management for Interactive Scene Navigation and Modification on Tiled Display Walls.

Simultaneous modification and navigation of massive 3D models are difficult because repeated data edits affect the data layout and coherency on a secondary storage, which in turn affect the interactive out-of-core rendering performance. In this paper, we propose a novel approach for distributed data management for simultaneous interactive navigation and modification of massive 3D data using the readily available infrastructure of a tiled display. Tiled multi-displays, projection or LCD panel based, driven by a PC cluster, can be viewed as a cluster of storage-compute-display (SCD) nodes. Given a cluster of SCD node infrastructure, we first propose a distributed memory hierarchy for interactive rendering applications. Second, in order to further reduce the latency in such applications, we propose a new data partitioning approach for distributed storage among the SCD nodes that reduces the variance in the data load across the SCD nodes. Our data distribution method takes in a data set of any size, and reorganizes it into smaller partitions, and stores it across the multiple SCD nodes. These nodes store, manage, and coordinate data with other SCD nodes to simultaneously achieve interactive navigation and modification. Specifically, the data is not duplicated across these distributed secondary storage devices. In addition, coherency in data access, due to screen-space adjacency of adjacent displays in the tile, as well as object space adjacency of the data sets, is well leveraged in the design of the data management technique. Empirical evaluation on two large data sets, with different data density distribution, demonstrates that the proposed data management approach achieves superior performance over alternative state-of-the-art methods.

Global SH-wave propagation in a 2D whole Moon model using the parallel hybrid PSM/FDM method

We present numerical modeling of SH-wave propagation for the recently proposed whole Moon model and try to improve our understanding of lunar seismic wave propagation. We use a hybrid PSM/FDM method on staggered grids to solve the wave equations and implement the calculation on a parallel PC cluster to improve the computing efficiency. Features of global SH-wave propagation are firstly discussed for a 100-km shallow and 900-km deep moonquakes, respectively. Effects of frequency range and lateral variation of crust thickness are then investigated with various models. Our synthetic waveforms are finally compared with observed Apollo data to show the features of wave propagation that were produced by our model and those not reproduced by our models. Our numerical modeling show that the low-velocity upper crust plays significant role in the development of reverberating wave trains. Increasing frequency enhances the strength and duration of the reverberations. Surface multiples dominate wavefields for shallow event. Core–mantle reflections can be clearly identified for deep event at low frequency. The layered whole Moon model and the low-velocity upper crust produce the reverberating wave trains following each phases consistent with observation. However, more realistic Moon model should be considered in order to explain the strong and slow decay scattering between various phases shown on observation data.

Implicit predictor–corrector central finite difference scheme for the equations of magnetohydrodynamic simulations

Nowadays most of supercomputers are based on the frame of PC cluster; therefore, the efficiency of parallel computing is of importance especially with the increasing computing scale. This paper proposes a high-order implicit predictor–corrector central finite difference (iPCCFD) scheme and demonstrates its high efficiency in parallel computing. Of special interests are the large scale numerical studies such as the magnetohydrodynamic (MHD) simulations in the planetary magnetosphere. An iPCCFD scheme is developed based on fifth-order central finite difference method and fourth-order implicit predictor–corrector method in combination with elimination-of-the-round-off-errors (ERE) technique. We examine several numerical studies such as one-dimensional Brio–Wu shock tube problem, two-dimensional Orszag–Tang vortex system, vortex type K–H instability, kink type K–H instability, field loop advection, and blast wave. All the simulation results are consistent with many literatures. iPCCFD can minimize the numerical instabilities and noises along with the additional diffusion terms. All of our studies present relatively small numerical errors without employing any divergence-free reconstruction. In particular, we obtain fairly stable results in the two-dimensional Brio–Wu shock tube problem which well conserves ∇⋅B=0 throughout the simulation. The ERE technique removes the accumulation of roundoff errors in the uniform or non-disturbed system. We have also shown that iPCCFD is characterized by the high order of accuracy and the low numerical dissipation in the circularly polarized Alfvén wave tests. The proposed iPCCFD scheme is a parallel-efficient and high precision numerical scheme for solving the MHD equations in hyperbolic conservation systems.

An application of partition method for solving 3D Stokes equation

In our previous work we have studied the performance of a parallel algorithm, based on a direction splitting approach, for solving of time dependent Stokes equation. We used a rectangular uniform mesh, combined with a central difference scheme for the second derivatives. Hence, the proposed algorithm required only solution of tridiagonal linear systems.In our work, we are targeting massively parallel computers, as well as clusters of multi-core nodes. The somehow slower (experimentally-established) performance of the proposed approach was observed when using all cores on a single node of a cluster. To remedy this problem, we tried to use LAPACK subroutines from the multi-threaded layer library, but the parallel performance of the code (while improved) was still not satisfactory on a single (multi-core) node.Our current work considers hybrid parallelization based on the MPI and OpenMP standards. It is motivated by the need to maximize the parallel efficiency of our implementation of the proposed algorithm. Essential improvements of the parallel algorithm are achieved by introducing two levels of parallelism: (i) between-node parallelism based on the MPI and (ii) inside-node parallelism based on the OpenMP. The implementation was tested on Linux clusters with Intel processors and on the IBM supercomputer.

Robustness Analysis of an Aircraft Design for Short Takeoff and Landing

As part of the Collaborative Research Center 880, preliminary aircraft design activities are carried out for a new class of low-noise cruise-efficient short takeoff and landing (CESTOL) transport aircraft. A corresponding aircraft is quite different from a state-of-the-art commercial aircraft because of the use of a high-lift system with active flow control. The fact that new technologies are not sufficiently understood yet in combination with the assumption of common design data and the use of classical calculation methods expresses itself in uncertainties that are of epistemic character. The robustness of a deterministic CESTOL aircraft design toward parameters such as the necessary engine thrust, direct operating costs, and the maximum takeoff and landing distances is investigated here concerning the mentioned uncertainties. For this purpose, a stochastic description of parameter variations of the design is formulated. Stochastic quantities are computed by Monte Carlo sampling to rate the robustness. A distributed component-based software implementation is used to perform the Monte Carlo sampling. The software system is installed on a Linux cluster with several multi-CPU computers; a deterministic sample is simulated through the design program PrADO.

Three-Dimensional Numerical Simulation of an External Gear Pump With Decompression Slot and Meshing Contact Point

Recently several works have been published on numerical simulation of an external gear pump (EGP). Such kinds of pumps are simple and relatively inexpensive, and are frequently used in fluid power applications, such as fluid power in aeronautical, mechanical, and civil engineering. Nevertheless, considerable effort is being undertaken to improve efficiency and reduce noise and vibration produced by the flow and pressure pulsations. Numerical simulation of an EGP is not straightforward principally for two main reasons. First, the gearing mechanism between gears makes it difficult to handle a dynamic mesh without a considerable deterioration of mesh quality. Second, the dynamic metal–metal contact simulation is important when high pressure outflow has to be reproduced. The numerical studies published so far are based on a two-dimensional (2D) approximation. The aim of the present work is to contribute to the understanding of the fluid flow inside an EGP by means of a complete three-dimensional (3D) parallel simulation on a cluster. The 3D flow is simulated in a linux cluster with a solver developed with the openfoam Toolbox. The hexahedral mesh quality is maintained by periodically replacing the mesh and interpolating the physical magnitudes fields. The meshing contact point is simulated with the viscous wall approach, using a viscosity model based on wall proximity. The results for the flow rate ripples show a similar behavior to that obtained with 2D simulations. However, the flow presents important differences inside the suction and the discharge chambers, principally in the regions of the pipes' connection. Moreover, the decompression slot below the gearing zone, which can not be simulated with a 2D approximation, enables a more realistic simulation of a contact ratio greater than 1. The results are compared with experimental measurements recently published.

The Design of the Plasma Control System in KTX**supported by the External Cooperation Program of BIC, Chinese Academy of Sciences (No. GJHZ201303), the National Magnetic Confinement Fusion Research Program of China (Nos. 2014GB10300, 2012GB105000 and 2011GB10600), National Natural Science Foundation of China (No. 11205200), the Anhui Provincial Science Foundation for Outstanding Young Talent of China (No. 2012SQRL265)

KTX (Keda Torus for eXperiment) is a new reversed field pinch device. The KTX plasma control system (PCS) can provide real-time, stable, flexible plasma control which is designed by ASIPP (Institute of Plasma Physics, Chinese Academy of Sciences), based on the Linux cluster system and EPICS (Experimental Physics and Industrial Control System) framework, and developed from DIII-D (Doublet III-D) PCS. The control of the equilibrium field in KTX uses a PID (Proportional-Integral-Derivative) feedback controller. The control of the gas injection is an open loop control. The plasma control simulation system is one part of the plasma control system, which is used to test the plasma control algorithm if it is revised and updated. The KTX PCS has been successfully tested using HT-7 (Hefei Torus 7) experiment data in simulation mode. In the next phase, an error field feedback control and KTX simulator will be added to the KTX PCS, and the KTX PCS will be applied in experiments in the future.

Synthesizing MPI Implementations from Functional Data-Parallel Programs

Distributed memory architectures such as Linux clusters have become increasingly common but remain difficult to program. We target this problem and present a novel technique to automatically generate data distribution plans, and subsequently MPI implementations in C++, from programs written in a functional core language. The main novelty of our approach is that we support distributed arrays, maps, and lists in the same framework, rather than just arrays. We formalize distributed data layouts as types, which are then used both to search (via type inference) for optimal data distribution plans and to generate the MPI implementations. We introduce the core language and explain our formalization of distributed data layouts. We describe how we search for data distribution plans using an adaptation of the Damas---Milner type inference algorithm, and how we generate MPI implementations in C++ from such plans.

Parallel Tree Accumulations on MapReduce

MapReduce is a remarkable parallel programming model as well as a parallel processing infrastructure for large-scale data processing. Since it is now widely available on cloud environments, developing methodology or patterns for MapReduce programming is important. In particular, XML is the de facto standard for representing data, and processing semi-structured data is involved in many applications. The target computational patterns in this paper are tree accumulations. Tree accumulations are shape-preserving computations over a tree in which values are updated through flows over the tree. We develop BSP algorithms for two tree accumulations as extensions of the BSP algorithm for tree reduction by Kakehi et al. (Tech. Rep. METR 2006-64, Department of Mathematical Informatics, Graduate School of Information Science and Technology, The University of Tokyo, 2006). We also implemented the two-superstep algorithms with a single MapReduce execution. Experimental results on a 16-node PC cluster show good speedups of a factor of 10.9---12.7.

Cluster computing for the large scale discrete fractional Cable equation

This paper presents a numerical simulation technique for the fractional Cable equation in large scale domain. Special attention is given to the parallel execution of the fractional weighted average finite difference method (FWA-FDM) on distributed system with explicit message passing, where the fractional derivative is defined in Riemann–Liouville sense. The resultant huge system of equations is studied using precondition conjugate gradient method (PCG), with the implementation of cluster computing on it. The proposed approach fulfills the suitability for the implementation on Linux PC cluster through the minimization of inter-process communication. To examine the efficiency and accuracy of the proposed method, numerical test experiments using different number of the Linux PC cluster nodes are studied. The performance metrics clearly show the benefit of using the proposed approach on the Linux PC cluster in terms of execution time reduction and speedup with respect to the sequential running in a single PC.