Multi-node Cluster Research Articles

Scaling machine learning for target prediction in drug discovery using Apache Spark

In the context of drug discovery, a key problem is the identification of candidate molecules that affect proteins associated with diseases. Inside Janssen Pharmaceutica, the Chemogenomics project aims to derive new candidates from existing experiments through a set of machine learning predictor programs, written in single-node C++. These programs take a long time to run and are inherently parallel, but do not use multiple nodes. We show how we reimplemented the pipeline using Apache Spark, which enabled us to lift the existing programs to a multi-node cluster without making changes to the predictors. We have benchmarked our Spark pipeline against the original, which shows almost linear speedup up to 8 nodes. In addition, our pipeline generates fewer intermediate files while allowing easier checkpointing and monitoring.

Storm Analysis with Raw Rainfall Dataset by using Artificial Neural Network and Min-Max Algorithms

Objectives: The main objective of this paper is to propose a methodology to develop a Storm analysis model from Raw Rainfall dataset using techniques such as Artificial Neural Network and Min-Max Algorithm. Storm analysis model aims at predicting the occurrence and strength of a storm by analyzing the rainfall data of that region. Methods: In the proposed methodology the raw rainfall dataset is being trained by Artificial Neural network based on the three layers -Input, Hidden, and Output layers. The trained dataset is then summarized into a model which performs the prediction of storm centric characteristics. Neural network training is implemented in Hadoop framework. We obtain a considerable improvement in the total performance of the system by employing Artificial Neural Network. Min-Max algorithm is also used in the system for predicting the intensity of storm. The dataset used for training and prediction consists of daily rainfall data of Cherrapunjee area collected by The Meteorological Department of India. Findings: In the existing system, the raw rainfall dataset is collected and stored in a relational database and then map-reduce based techniques are applied for storm analysis. The major disadvantages associated with this technique are the performance and accuracy rate get reduced with increase in data size. In the proposed methodology as the raw rainfall dataset is being trained by Artificial Neural network the performance and accuracy rate got improved. Also, the training process is done on multi-node hadoop cluster by considering large raw rainfall dataset. With multi-node hadoop cluster there was a large reduction in the total training time. Storm depth of a particular region is calculated by applying MIN-MAX algorithm. This improved the total efficiency of the storm intensity prediction. Applications/Improvement: The performance of the system can be further improved by reducing the training time by adding more nodes while implementing the process in multi node hadoop cluster. Also higher prediction accuracy can be obtained by combining various suitable fuzzy inference models5 with the proposed neural network mode.

HadoopCL2: Motivating the Design of a Distributed, Heterogeneous Programming System With Machine-Learning Applications

Machine learning (ML) algorithms have garnered increased interest as they demonstrate improved ability to extract meaningful trends from large, diverse, and noisy data sets. While research is advancing the state-of-the-art in ML algorithms, it is difficult to drastically improve the real-world performance of these algorithms. Porting new and existing algorithms from single-node systems to multi-node clusters, or from architecturally homogeneous systems to heterogeneous systems, is a promising optimization technique. However, performing optimized ports is challenging for domain experts who may lack experience in distributed and heterogeneous software development. This work explores how challenges in ML application development on heterogeneous, distributed systems shaped the development of the HadoopCL2 (HCL2) programming system. ML applications guide this work because they exhibit features that make application development difficult: large & diverse datasets, complex algorithms, and the need for domain-specific knowledge. The goal of this work is a general, MapReduce programming system that outperforms existing programming systems. This work evaluates the performance and portability of HCL2 against five ML applications from the Mahout ML framework on two hardware platforms. HCL2 demonstrates speedups of greater than 20x relative to Mahout for three computationally heavy algorithms and maintains minor performance improvements for two I/O bound algorithms.

A Service Oriented Framework for Analysing Social Network Activities

Analysing and monitoring Social Networking activities raise multiple challenges for the evolution of Service Oriented Systems Engineering. This is particularly evident for event detection in social networks and, more in general, for large-scale Social Analytics, which require continuous processing of data. In this paper we present a service oriented framework exploring effective ways to leverage the opportunities coming from innovations and evolutions in computational power, storage, and infrastructures, with particular focus on modern architectures including in-memory database technology, in-database computation, massive parallel processing, Open Data Services, and scalability with multi-node clusters in Cloud. A prototype of this system was experimented in the contest of a specific kind of social event, an art exhibition of sculptures, where the system collected and analyzed in real-time the tweets issued in an entire region, including exhibition sites, and continuously updated analytical dashboards placed in one of the exhibition rooms.

Efficient Pairwise Document Similarity Computation in Big Datasets

Support Vector Machine (SVM) is extremely powerful and widely accepted classifier in the field of machine learning due to its better generalization capability. However, SVM is not suiTable for large scale dataset due to its high computational complexity. The computation and storage requirement increases tremendously for large dataset. In this paper, we have proposed a MapReduce based SVM for large scale data. MapReduce is a distributed programming model which works on large scale dataset by dividing the huge datasets in smaller chunks. MapReduce distribution model works on several frame works like Hadoop Twister and so on. In this paper, we have analyzed the impact of penalty and kernel parameters on the performance of parallel SVM. The experimental result shows that the number of support vectors and predictive accuracy of SVM is affected by the choice of these parameters. From experimental results, it is also analyzed that the computation time taken by the SVM with multi-node cluster is less as compared to the single node cluster for large dataset.

Time-domain BEM for the wave equation on distributed-heterogeneous architectures: A blocking approach

We parallelize the TD-BEM on clusters of multicore nodes enhanced with multiple GPU.SpMV is replaced by a more efficient operator by reordering the computation.GPU kernels achieve high Flop-rate using blocking schemes.Idle time is drastically reduced by using a greedy balancing algorithm. The problem of time-domain BEM for the wave equation in acoustics and electromagnetism can be expressed as a sparse linear system composed of multiple interaction/convolution matrices. It can be solved by using sparse matrix-vector products which are inefficient to achieve high Flop-rate neither on CPUs nor GPUs. In this paper we extend the approach proposed in a previous work 1 in which we re-order the computation to get a special matrix structure with one dense vector per row. This new structure is called a slice matrix and is computed with a custom matrix/vector product operator. In this study, we present an optimized implementation of this operator on Nvidia GPUs based on two blocking strategies. We explain how we can obtain multiple block-values from a slice and how these can be computed efficiently on GPUs since we target heterogeneous nodes composed of CPUs and GPUs. In order to deal with different efficiencies of the processing units we use a greedy heuristic that dynamically balances work among the workers. We demonstrate the performance of our system by studying the quality of the balancing heuristic and the sequential Flop-rate of the blocked implementations. Finally, we validate our implementation with an industrial test case on 8 heterogeneous nodes, each composed of 12 CPUs and 3 GPUs.

An application of partition method for solving 3D Stokes equation

In our previous work we have studied the performance of a parallel algorithm, based on a direction splitting approach, for solving of time dependent Stokes equation. We used a rectangular uniform mesh, combined with a central difference scheme for the second derivatives. Hence, the proposed algorithm required only solution of tridiagonal linear systems.In our work, we are targeting massively parallel computers, as well as clusters of multi-core nodes. The somehow slower (experimentally-established) performance of the proposed approach was observed when using all cores on a single node of a cluster. To remedy this problem, we tried to use LAPACK subroutines from the multi-threaded layer library, but the parallel performance of the code (while improved) was still not satisfactory on a single (multi-core) node.Our current work considers hybrid parallelization based on the MPI and OpenMP standards. It is motivated by the need to maximize the parallel efficiency of our implementation of the proposed algorithm. Essential improvements of the parallel algorithm are achieved by introducing two levels of parallelism: (i) between-node parallelism based on the MPI and (ii) inside-node parallelism based on the OpenMP. The implementation was tested on Linux clusters with Intel processors and on the IBM supercomputer.

English

Plagiarism of digital documents that happened frequently should be prevented and reduced. In order to measure the degree of document's similarity, Ferret Algorithm is used on MapReduce and Hadoop framework. This application program is tested using ten pair documents in English and ten pair documents in Indonesian to get the degree of document's similarity between document's pairs. In this experiment, it was also tested the accuracy of the system by using several kind of n- grams, that is 1-gram, 2-gram, 3-gram, 4-gram, 5-gram, and 6-gram. In addition, this experiment is conducted on stand-alone computers and Hadoop multi-node cluster system. In order to compare the time taken of the system, it is used nine pairs of documents with various size of documents.

Comparison of Machine Learning Algorithm on Map Reduction for Performance Improvement in Big Data

Background: As massive data acquisition and storage becomes increasingly affordable, a wide variety of enterprises are engaged in sophisticated data analysis. The amount of digital information which is majorly unstructured produced is exceeding day by day. Method: MapReduce programming method is easily applicable to many different learning algorithms. Machine Learning is at the core of data analysis. Traditional machine learning algorithms speed up at a time to fit the statistical query model on multicore computers. Data Sharing is avoided by Hadoop whereas Machine learning Algorithm needs data to be stored in single place. The method compares the machine learning algorithms on MapReduce paradigm for evaluating speed. Findings: MapReduce programming model enables easy development of scalable parallel applications to process large clusters of data. Hadoop Distributed File System runs the MapReduce jobs which influence the performance significantly while handling huge data set stored on different nodes of a multi node cluster. This paper analyses on developing machine learning algorithms on Hadoop to process large clusters of data. Analyzing logistic regression algorithms on MapReduce for evaluating the performance to speed up processing by developing a cost model. The attributes of the system are evaluated for improving time efficiency. The objective is to provide ad hoc performance for MapReduce programs which run on large data sets. Improvements: A method for optimizing job assignment on machine learning is implemented in order to minimize the total execution time. This feature improves the productivity of MapReduce users to optimize performance efficiency.

Using distributed memory parallel computers and GPU clusters for multidimensional Monte Carlo integration

SummaryThe aim of this paper is to show that the multidimensional Monte Carlo integration can be efficiently implemented on various distributed memory parallel computers and clusters of multicore nodes using recently developed parallel versions of linear congruential generator and lagged Fibonacci generator pseudorandom number generators. We show how to accelerate the overall performance by offloading some computations to Graphics Processing Units (GPUs), and we discuss how to transform Message Passing Interface (MPI) + OpenMP programs to MPI + OpenMP + CUDA model. We explain how to utilize multiple cores of CPUs together with multiple GPU accelerators within a single node and how to achieve reasonable load balancing of all computational resources of GPU‐accelerated multicore nodes. We present and discuss the results of experiments performed on the following target architectures: IBM Blue Gene/Q parallel computer, a cluster of Intel Xeon E5‐2660 servers, and a Tesla‐based GPU cluster with Intel Xeon X5650 multicore processors. The results are presented from two points of view: strong scaling and weak scaling. We also compare the performance of all considered architectures. Copyright © 2014 John Wiley & Sons, Ltd.

Applying a new synchronous inversion of seismograms using maximum likelihood method and stochastic refinements to study ultra-thin oil-saturated reservoirs

Seismic inversion makes use of single or multi-dimensional seismic trace data to derive elastic properties from seismic amplitude data. These elastic properties are subsequently used to make inferences about seismic lithology, lithologic properties, and the presence of fluids. Most implementations of seismic inversions operate on limited stacked subsets of the seismic data. Common inputs to inversion methods include angle stacks, offset stacks, or multiple stacks of time-lapsed data. The implementation in this paper makes use of a synchronous pre-stack inversion based on the maximum likelihood algorithm (Anat Canning and Alex Malkin, 2009). The PMLI inversion is applicable to time and depth angle or offset seismograms. It is a full pre-stack inversion, which inverts each and every trace of an offset or angle gather and returns elastic properties. The PMLI inversion can also invert partial angular or offset sums, which can be created in any processing system. As a complete pre-stack inversion, PMLI provides higher accuracy and noise immunity as observed in the output results of Pw and Sw impedances, density, Vp/Vs, Poisson’s ratio, Lambda Rho (LR) and Mu Rho (MR). PMLI also incorporates a necessary and effective preprocessing step, which is performed on-the-fly as part of the inversion process. Because the PMLI method operates on gathers, it is easily and efficiently parallelized for multi-core and multi-node clusters and for shared memory architectures.

Cluster synchronization induced by one-node clusters in networks with asymmetric negative couplings

This paper deals with the problem of cluster synchronization in networks with asymmetric negative couplings. By decomposing the coupling matrix into three matrices, and employing Lyapunov function method, sufficient conditions are derived for cluster synchronization. The conditions show that the couplings of multi-node clusters from one-node clusters have beneficial effects on cluster synchronization. Based on the effects of the one-node clusters, an effective and universal control scheme is put forward for the first time. The obtained results may help us better understand the relation between cluster synchronization and cluster structures of the networks. The validity of the control scheme is confirmed through two numerical simulations, in a network with no cluster structure and in a scale-free network.

Parallel contributing area calculation with granularity control on massive grid terrain datasets

The calculation of contributing areas from digital elevation models (DEMs) is one of the important tasks in digital terrain analysis (DTA). The computational process usually involves two steps in a real application: (1) calculating flow directions via a flow model, and (2) computing the contributing area for each grid cell in the DEM. The traditional algorithm for calculating contributing areas is coded as a sequential program executed on a single processor. With the increase of scope and resolution of DEMs, the serial algorithm has become increasingly difficult to perform and is often very time-consuming, especially for DEMs of large areas and fine scales. In recent years, parallel computing is able to meet this challenge with the development of computer technology. However, the parallel implementation with granularity control, an efficient strategy to reap the best parallel performance and to break the limitation of computing resources in processing massive grid terrain datasets, has not been found in DTA research field. This paper develops a message-passing-interface (MPI) parallel approach with granularity control to calculate contributing areas. According to the proposed parallelization strategy, the parallel D8 algorithm with granularity control is designed as well as the parallel AreaD8 algorithm. Based on the domain decomposition of DEM data, it is possible for each process to process multiple partitions decomposed under a grain size. According to an iterative procedure of reading source data, executing the operator and writing resulting data, the partitions achieve the calculation results one by one in each process. The experimental results on a multi-node cluster show that the proposed parallel algorithms with granularity control are the powerful tools to process the big dataset and the parallel D8 algorithm is insensitive to granularity, while the parallel AreaD8 algorithm has an optimal grain size to reap the best parallel performance.

High-performance computing selection of models of DNA substitution for multicore clusters

This paper presents the high-performance computing (HPC) support of jModelTest2, the most popular bioinformatic tool for the statistical selection of models of DNA substitution. As this can demand vast computational resources, especially in terms of processing power, jModelTest2 implements three parallel algorithms for model selection: (1) a multithreaded implementation for shared memory architectures; (2) a message-passing implementation for distributed memory architectures, such as clusters; and (3) a hybrid shared/distributed memory implementation for clusters of multicore nodes, combining the workload distribution across cluster nodes with a multithreaded model optimization within each node. The main limitation of the shared and distributed versions is the workload imbalance that generally appears when using more than 32 cores, a direct consequence of the heterogeneity in the computational cost of the evaluated models. The hybrid shared/distributed memory version overcomes this issue reducing the workload imbalance through a thread-based decomposition of the most costly model optimization tasks. The performance evaluation of this HPC application on a 40-core shared memory system and on a 528-core cluster has shown high scalability, with speedups of the multithreaded version of up to 32, and up to 257 for the hybrid shared/distributed memory implementation. This can represent a reduction in the execution time of some analyses from 4 days down to barely 20 minutes. The implementation of the three parallel execution strategies of jModelTest2 presented in this paper are available under a GPL license at http://code.google.com/jmodeltest2.

Hierarchical QR factorization algorithms for multi-core clusters

This paper describes a new QR factorization algorithm which is especially designed for massively parallel platforms combining parallel distributed nodes, where a node is a multi-core processor. These platforms represent the present and the foreseeable future of high-performance computing. Our new QR factorization algorithm falls in the category of the tile algorithms which naturally enables good data locality for the sequential kernels executed by the cores (high sequential performance), low number of messages in a parallel distributed setting (small latency term), and fine granularity (high parallelism). Each tile algorithm is uniquely characterized by its sequence of reduction trees. In the context of a cluster of nodes, in order to minimize the number of inter-processor communications (aka, “communication-avoiding”), it is natural to consider hierarchical trees composed of an “inter-node” tree which acts on top of “intra-node” trees. At the intra-node level, we propose a hierarchical tree made of three levels: (0) “TS level” for cache-friendliness, (1) “low-level” for decoupled highly parallel inter-node reductions, (2) “domino level” to efficiently resolve interactions between local reductions and global reductions. Our hierarchical algorithm and its implementation are flexible and modular, and can accommodate several kernel types, different distribution layouts, and a variety of reduction trees at all levels, both inter-node and intra-node. Numerical experiments on a cluster of multi-core nodes (i) confirm that each of the four levels of our hierarchical tree contributes to build up performance and (ii) build insights on how these levels influence performance and interact within each other. Our implementation of the new algorithm with the DAGuE scheduling tool significantly outperforms currently available QR factorization software for all matrix shapes, thereby bringing a new advance in numerical linear algebra for petascale and exascale platforms.

Large-Scale Modeling of Epileptic Seizures: Scaling Properties of Two Parallel Neuronal Network Simulation Algorithms

Our limited understanding of the relationship between the behavior of individual neurons and large neuronal networks is an important limitation in current epilepsy research and may be one of the main causes of our inadequate ability to treat it. Addressing this problem directly via experiments is impossibly complex; thus, we have been developing and studying medium-large-scale simulations of detailed neuronal networks to guide us. Flexibility in the connection schemas and a complete description of the cortical tissue seem necessary for this purpose. In this paper we examine some of the basic issues encountered in these multiscale simulations. We have determined the detailed behavior of two such simulators on parallel computer systems. The observed memory and computation-time scaling behavior for a distributed memory implementation were very good over the range studied, both in terms of network sizes (2,000 to 400,000 neurons) and processor pool sizes (1 to 256 processors). Our simulations required between a few megabytes and about 150 gigabytes of RAM and lasted between a few minutes and about a week, well within the capability of most multinode clusters. Therefore, simulations of epileptic seizures on networks with millions of cells should be feasible on current supercomputers.

Research on MPI-Based Parallel Max-Min Ant System

When solving large scale combinatorial optimization problems, Max-Min Ant System requires long computation time. MPI-based Parallel Max-Min Ant System described in this paper can ensure the quality of the solution, as well as reduce the computation time. Numerical experiments on the multi-node cluster system show that when solving the traveling salesman problem, MPI-based Parallel Max-Min Ant System can get better computational efficiency.

Repair Method of Complex Network Based on Matthew Effect

Complex network repair after suffering the deliberate assault becomes extraordinarily important. In this paper, a repair method of complex network based on Matthew Effect was proposed. Single-node selective attack algorithm and multi-node cluster attack algorithm was given. Aiming at the two kinds of attack, linear detection algorithm and BA network generation algorithm was put forward to get experiment data. Correspondingly, repair experiments were done. Experimental results show that repair rate of the method is more than 95% in sampling Internet and BA network. For repair rate of complex network, the conception of stability and its mathematics description was addressed. Experiments show that the complex network can achieve a steady topology state after some steps of attacks and repairs.

Fast anomaly detection in hyperspectral images with RX method on heterogeneous clusters

Remotely sensed hyperspectral sensors provide image data containing rich information in both the spatial and the spectral domain, and this information can be used to address detection tasks in many applications. One of the most widely used and successful algorithms for anomaly detection in hyperspectral images is the RX algorithm. Despite its wide acceptance and high computational complexity when applied to real hyperspectral scenes, few approaches have been developed for parallel implementation of this algorithm. In this paper, we evaluate the suitability of using a hybrid parallel implementation with a high-dimensional hyperspectral scene. A general strategy to automatically map parallel hybrid anomaly detection algorithms for hyperspectral image analysis has been developed. Parallel RX has been tested on an heterogeneous cluster using this routine. The considered approach is quantitatively evaluated using hyperspectral data collected by the NASA’s Airborne Visible Infra-Red Imaging Spectrometer system over the World Trade Center in New York, 5 days after the terrorist attacks. The numerical effectiveness of the algorithms is evaluated by means of their capacity to automatically detect the thermal hot spot of fires (anomalies). The speedups achieved show that a cluster of multi-core nodes can highly accelerate the RX algorithm.

On the Scaling Behavior of HDA*

HDA* is a simple, parallelization of A* where work is asynchronously distributed among the nodes by a global hash function. Using up to 1024 cores on a large distributed memory cluster, we evaluate HDA* for a domain-independent planner as well an application-specific 24-puzzle solver. We show that HDA* scales fairly well on a large cluster using up to 1024 cores. Our analysis of the scaling behavior shows that on a cluster of multicore nodes, using only a subset of the available cores and leaving some cores idle can, surprisingly, lead to better results.