Parallel Programming Model Research Articles

Bi‐criteria formulation for green scheduling with unrelated parallel machines with sequence‐dependent setup times

AbstractGiven the important role of machine scheduling in manufacturing industry, we discuss power consumption in sequencing jobs in a scheduling problem, assuming variable speed operation in machines. The problem involves defining the allocation of jobs to machines, the order of processing jobs and the speed of processing each job in each machine. This problem can be viewed as a type of green scheduling problem, dealing with sustainable use of energy consumption and environmental effects. We propose a mixed integer linear programming (MILP) model for the unrelated parallel machine‐scheduling problem with sequence‐dependent setup times, with independent and non‐preemptible jobs, minimizing the makespan and the total consumption of electricity. Furthermore, we employ a novel math‐heuristic algorithm, named multi‐objective smart pool search matheuristic (or simply smart pool), for finding solutions near the Pareto front, in a restricted computational budget. As a case study, a new set of instances is created for the problem. Those instances are solved using the classical ε‐constrained method and the smart pool method. The obtained sets of non‐dominated solutions indicate the conflict between both objectives, highlighting the relevance of the suggested approach to industry. From the obtained results, it was verified that the smart pool achieved good convergence towards the true Pareto front, as indicated by the hyper‐volume metric, presenting lower average time for finding solutions on the Pareto front. In small to medium size instances, the smart pool search method can achieve very good approximations of the Pareto front with less computational effort than traditional methods.

Dynamic Work Load Balancing for Compute Intensive Application Using Parallel and Hybrid Programming Models on CPU-GPU Cluster

Dynamic Work Load Balancing for Compute Intensive Application Using Parallel and Hybrid Programming Models on CPU-GPU Cluster

A Comparative Study of Join Algorithms in Mapreduce

To analyze large volumes of data, a set of techniques is present in the IT community as the MapReduce paradigm, parallel RDBMS, column storage, and combinations of those technics. MapReduce is a parallel programming model introduced by Google, which enables an easy parallelization of tasks while hiding the details and the complexity of parallel computations on very large datasets across a large number of machines. Our study will be concerned about the MapReduce Data Analytics, the most important data analysis treatments in MapReduce is the treatment of the Logs files as in the case of web applications, which can be in the form of selection operation, aggregation, or filtering, the most useful and expensive operation is that of the join, the logs files will often need to be joined with one or more table references, however the MapReduce paradigm is not designed to process multiple inputs. While processing relational data is a common need, this limitation causes difficulties and inefficiency when MapReduce is applied on relational operations like joins. The aim of this paper is to compare a number of wellknown join strategies in MapReduce, analyze the costs associated to a MapReduce program in terms of I/O and CPU used, and present some techniques of optimizations from related works.

Importance of MapReduce for Big Data Applications: A Survey

Significant regard for MapReduce framework has been trapped by a wide range of areas. It is presently a practical model for data-focused applications because of its basic interface of programming, high elasticity, and capacity to withstand the subjection to defects. Additionally, it is fit for preparing a high extent of data in Distributed Computing environments (DCE). MapReduce, on various events, has turned out to be material to a wide scope of areas. MapReduce is a parallel programming model and a related usage presented by Google. In the programming model, a client determines the calculation by two capacities, Map and Reduce. The basic MapReduce library consequently parallelizes the calculation and handles muddled issues like data dispersion, load adjusting, and adaptation to non-critical failure. Huge data spread crosswise over numerous machines, need to parallelize. Moves the data, and gives booking, adaptation to non-critical failure. A writing survey on the MapReduce programming in different areas has completed in this paper. An examination course has been distinguished by utilizing a writing audit.

Incremental parallel computing for continuous queries in dynamic graphs using a transactional model

SummaryIn dynamically evolving graphs, one may be interested in continuously observing certain properties of the graph. One approach for continuous monitoring is to re‐execute the graph analytics program on the entire graph after it is updated. However, this approach can lead to high computation cost and latency in case of large graphs. An alternate approach, which we present in this paper, is to execute the analytics program only initially and then perform incremental computations for supporting continuous queries as the graph is modified. The goal of our work is to develop incremental parallel computing techniques to continuously monitor a graph as it is updated to check for the properties of interest. We present the results of our investigation of utilizing a transactional model of parallel programming for supporting continuous queries on dynamic and evolving graphs. In our model, the graph updates are performed as transactions, which trigger the execution of a set of transactional tasks to perform incremental computations. In our testbed system, the graph data is stored in the RAM of cluster nodes, and continuous queries involve the parallel execution of transactional tasks on the cluster nodes. Using a set of graph problems, we illustrate this approach and evaluate its performance.

Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review

While modern parallel computing systems offer high performance, utilizing these powerful computing resources to the highest possible extent demands advanced knowledge of various hardware architectures and parallel programming models. Furthermore, optimized software execution on parallel computing systems demands consideration of many parameters at compile-time and run-time. Determining the optimal set of parameters in a given execution context is a complex task, and therefore to address this issue researchers have proposed different approaches that use heuristic search or machine learning. In this paper, we undertake a systematic literature review to aggregate, analyze and classify the existing software optimization methods for parallel computing systems. We review approaches that use machine learning or meta-heuristics for software optimization at compile-time and run-time. Additionally, we discuss challenges and future research directions. The results of this study may help to better understand the state-of-the-art techniques that use machine learning and meta-heuristics to deal with the complexity of software optimization for parallel computing systems. Furthermore, it may aid in understanding the limitations of existing approaches and identification of areas for improvement.

An enhanced un-split face-vertex flux-based VoF method

Multiple advancements of the dimensionally un-split, flux-based geometrical Volume-of-Fluid method on unstructured meshes are proposed. A second-order accurate interface reconstruction algorithm for both two-and-three dimensions is developed on unstructured hexahedral meshes. A new triangulation algorithm is developed that results in an absolute increase in accuracy and can be directly applied to other geometrical Volume-of-Fluid methods on both structured and unstructured meshes. A second-order accurate interpolation of point displacements is proposed for unstructured meshes. A global parallel error re-distribution algorithm is developed, with no negative effects on the L1 error convergence and computational costs. All algorithms are parallelized using the message passing parallel programming model, with a minimal parallel communication overhead. The results show L1 errors to be exactly numerically bounded, second-order convergent and lower than the errors reported for other contemporary methods, volume conservation that is near machine tolerance, as well as execution times comparable to those of methods developed on structured Cartesian meshes, regardless of the increased algorithmic complexity introduced by the connectivity of unstructured meshes.

A haoop-based parallel mining of frequent itemsets using N-Lists

Frequent pattern mining is the most important phase of association rule mining process because of its time and space complexity. Several methods have attempted to improve the performance of association rule mining by enhancing frequent pattern mining efficiency. Due to the large size of the data-sets and huge amounts of data which should be mined, many parallel and distributed mining approaches have been introduced to divide data-sets or to distribute mining processes between multiple processors or computers and thus, improve the efficiency of the mining process. In this paper, we propose a hadoop-based parallel implementation of PrePost+ algorithm for frequent itemset mining. In our parallel approach, the process of constructing N-Lists of itemsets has been distributed between the mappers and the operation of the final pruning process and extracting frequent itemsets has been carried out by reducers in a map-reduce parallel programming model. The experimental results show that our hadoop-based PrePost+(HBPrePost+) algorithm outperforms one of the best existing parallel methods of frequent itemset mining (PARMA) in terms of execution time.

Implicit in the development of parallel programming language using Visual Basic

Several studies have been performed to investigate the benefits of parallel programming models, and compared the models with same language family. The parallel technique is applicable only when the programmer has a single computer with multiple processors to perform various tasks individually or the programmer number of different computers connected by a single network. Parallel programming model in computing is an idea of parallel computer architecture. This research introduced the concept of parallel programming models and languages. The present research mainly focused on the different parallel programming models proposed by previous researchers using distinct programming languages. Through analyzing the strengths and weaknesses of various programming models used for parallel computing, a new model is developed. Along with this, the concept of OOP (Object-oriented programming) in the context of parallel computing is also discussed. This research proposed a model named as Asynchronous Dynamic load Balancing (ADLB). ADLB model used MPI as avehicle for library implementation rather as an end-user programming system.

A Parallel and Optimization Approach for Land-Surface Temperature Retrieval on a Windows-Based PC Cluster

Land-surface temperature (LST) is a very important parameter in the geosciences. Conventional LST retrieval is based on large-scale remote-sensing (RS) images where split-window algorithms are usually employed via a traditional stand-alone method. When using the environment to visualize images (ENVI) software to carry out LST retrieval of large time-series datasets of infrared RS images, the processing time taken for traditional stand-alone servers becomes untenable. To address this shortcoming, cluster-based parallel computing is an ideal solution. However, traditional parallel computing is mostly based on the Linux environment, while the LST algorithm developed within the ENVI interactive data language (IDL) can only be run in the Windows environment in our project. To address this problem, we combine the characteristics of LST algorithms with parallel computing, and propose the design and implementation of a parallel LST retrieval algorithm using the message-passing interface (MPI) parallel-programming model on a Windows-based PC cluster platform. Furthermore, we present our solutions to the problems associated with performance bottlenecks and fault tolerance during the deployment stage. Our results show that, by improving the parallel environment of the storage system and network, one can effectively solve the stability issues of the parallel environment for large-scale RS data processing.

On the adequacy of lightweight thread approaches for high-level parallel programming models

High-level parallel programming models (PMs) are becoming crucial in order to extract the computational power of current on-node multi-threaded parallelism. The most popular PMs, such as OpenMP or OmpSs, are directive-based: the complexity of the hardware is hidden by the underlying runtime system, improving coding productivity. The implementations of OpenMP usually rely on POSIX threads (pthreads), offering excellent performance for coarse-grained parallelism and a perfect match with the current hardware. OmpSs is a task oriented PM based on an ad hoc runtime solution called Nanos++; it is the precursor of the tasking parallelism in the OpenMP tasking specification. A recent trend in runtimes and applications points to leveraging massive on-node parallelism in conjunction with fine-grained and dynamic scheduling paradigms. In this paper we analyze the behavior of the OpenMP and OmpSs PMs on top of the recently emerged Generic Lightweight Threads (GLT) API. GLT exposes a common API for lightweight thread (LWT) libraries that offers the possibility of running the same application over different native LWT solutions. We describe the design details of those high-level PMs implemented on top of GLT and analyze different scenarios in order to assess where the use of LWTs may benefit application performance. Our work reveals those scenarios where LWTs overperform pthread-based solutions and compares the performance between an ad hoc solution and a generic implementation.

HitFlow: A Dataflow Programming Model for Hybrid Distributed- and Shared-Memory Systems

Dataflow programming consists in developing a program by describing its sequential stages and the interactions between them. The runtime systems supporting this kind of programming are responsible for exploiting the parallelism by concurrently executing the different stages as soon as their dependencies are met. In this paper we introduce a new parallel programming model and framework based on the dataflow paradigm. It presents a new combination of features that allows to easily map programs to shared or distributed memory, exploiting data locality and affinity to obtain the same performance than optimized coarse-grain MPI programs. These features include: It is a unique one-tier model that supports hybrid shared- and distributed-memory systems with the same abstractions; it can express activities arbitrarily linked, including non-nested cycles; it uses internally a distributed work-stealing mechanism to allow Multiple-Producer/Multiple-Consumer configurations; and it has a runtime mechanism for the reconfiguration of the dependences and communication channels which also allows the creation of task-to-task data affinities. We present an evaluation using examples of different classes of applications. Experimental results show that programs generated using this framework deliver good performance in hybrid distributed- and shared-memory environments, with a similar development effort as other dataflow programming models oriented to shared-memory.

Optimizing Apple Lossless Audio Codec Algorithm using NVIDIA CUDA Architecture

As majority of the compression algorithms are implementations for CPU architecture, the primary focus of our work was to exploit the opportunities of GPU parallelism in audio compression. This paper presents an implementation of Apple Lossless Audio Codec (ALAC) algorithm by using NVIDIA GPUs Compute Unified Device Architecture (CUDA) Framework. The core idea was to identify the areas where data parallelism could be applied and parallel programming model CUDA could be used to execute the identified parallel components on Single Instruction Multiple Thread (SIMT) model of CUDA. The dataset was retrieved from European Broadcasting Union, Sound Quality Assessment Material (SQAM). Faster execution of the algorithm led to execution time reduction when applied to audio coding for large audios. This paper also presents the reduction of power usage due to running the parallel components on GPU. Experimental results reveal that we achieve about 80-90% speedup through CUDA on the identified components over its CPU implementation while saving CPU power consumption.

Emotional modelling and classification of a large-scale collection of scene images in a cluster environment.

The development of network technology and the popularization of image capturing devices have led to a rapid increase in the number of digital images available, and it is becoming increasingly difficult to identify a desired image from among the massive number of possible images. Images usually contain rich semantic information, and people usually understand images at a high semantic level. Therefore, achieving the ability to use advanced technology to identify the emotional semantics contained in images to enable emotional semantic image classification remains an urgent issue in various industries. To this end, this study proposes an improved OCC emotion model that integrates personality and mood factors for emotional modelling to describe the emotional semantic information contained in an image. The proposed classification system integrates the k-Nearest Neighbour (KNN) algorithm with the Support Vector Machine (SVM) algorithm. The MapReduce parallel programming model was used to adapt the KNN-SVM algorithm for parallel implementation in the Hadoop cluster environment, thereby achieving emotional semantic understanding for the classification of a massive collection of images. For training and testing, 70,000 scene images were randomly selected from the SUN Database. The experimental results indicate that users with different personalities show overall consistency in their emotional understanding of the same image. For a training sample size of 50,000, the classification accuracies for different emotional categories targeted at users with different personalities were approximately 95%, and the training time was only 1/5 of that required for the corresponding algorithm with a single-node architecture. Furthermore, the speedup of the system also showed a linearly increasing tendency. Thus, the experiments achieved a good classification effect and can lay a foundation for classification in terms of additional types of emotional image semantics, thereby demonstrating the practical significance of the proposed model.

Efficient utilization of multi-core processors and many-core co-processors on supercomputer beacon for scalable geocomputation and geo-simulation over big earth data

Digital earth science data originated from sensors aboard satellites and platforms such as airplane, UAV, and mobile systems are increasingly available with high spectral, spatial, vertical, and temporal resolution data. When such big earth science data are processed and analyzed via geocomputation solutions, or utilized in geospatial simulation or modeling, considerable computing power and resources are necessary to complete the tasks. While classic computer clusters equipped by central processing units (CPUs) and the new computing resources of graphics processing units (GPUs) have been deployed in handling big earth data, coprocessors based on the Intel’s Many Integrated Core (MIC) Architecture are emerging and adopted in many high-performance computer clusters. This paper introduces how to efficiently utilize Intel’s Xeon Phi multicore processors and MIC coprocessors for scalable geocomputation and geo-simulation by implementing two algorithms, Maximum Likelihood Classification (MLC) and Cellular Automata (CA), on supercomputer Beacon, a cluster of MICs. Four different programming models are examined, including (1) the native model, (2) the offload model, (3) the symmetric model, and (4) the hybrid-offload model. It can be concluded that while different kinds of parallel programming models can enable big data handling efficiently, the hybrid-offload model can achieve the best performance and scalability. These different programming models can be applied and extended to other types of geocomputation to handle big earth data.

Toward Exascale Computing Systems: An Energy Efficient Massive Parallel Computational Model

The emerging Exascale supercomputing system expected till 2020 will unravel many scientific mysteries. This extreme computing system will achieve a thousand-fold increase in computing power compared to the current petascale computing system. The forthcoming system will assist system designers and development communities in navigating from traditional homogeneous to the heterogeneous systems that will be incorporated into powerful accelerated GPU devices beside traditional CPUs. For achieving ExaFlops (10^18 calculations per second) performance through the ultrascale and energy-efficient system, the current technologies are facing several challenges. Massive parallelism is one of these challenges, which requires a novel energy-efficient parallel programming (PP) model for providing the massively parallel performance. In the current study, a new parallel programming model has been proposed, which is capable of achieving massively parallel performance through coarse-grained and fine-grained parallelism over inter-node and intra-node architectural-based processing. The suggested model is a tri-level hybrid of MPI, OpenMP and CUDA that is computable over a heterogeneous system with the collaboration of traditional CPUs and energy-efficient GPU devices. Furthermore, the developed model has been demonstrated by implementing dense matrix multiplication (DMM). The proposed model is considered an initial and leading model for obtaining massively parallel performance in an Exascale computing system.

Editorial for the Special Issue on Parallel and Concurrent Functional Programming

Functional languages are uniquely suited to providing programmers with a programming model for parallel and concurrent computing. This is reflected in the wide range of work that is currently underway, both on parallel and concurrent functional languages, as well as on bringing functional language features to other programming languages. This has resulted in a rapidly growing number of practical applications. The Journal of Functional Programming decided to dedicate a special issue to this field to showcase the state of the art in how functional languages and functional concepts currently assist programmers with the task of managing the challenges of creating parallel and concurrent systems.

Evaluating attainable memory bandwidth of parallel programming models via BabelStream

Many scientific codes consist of memory bandwidth bound kernels. One major advantage of many-core devices such as general purpose graphics processing units (GPGPUs) and the Intel Xeon Phi is their focus on providing increased memory bandwidth over traditional CPU architectures. Peak memory bandwidth is usually unachievable in practice and so benchmarks are required to measure a practical upper bound on expected performance. We augment the standard STREAM kernels with a dot product kernel to investigate the performance of simple reduction operations on large arrays. The choice of programming model should ideally not limit the achievable performance on a device. BabelStream (formally GPU-STREAM) has been updated to incorporate a wide variety of the latest parallel programming models, all implementing the same parallel scheme. As such this tool can be used as a kind of Rosetta Stone which provides both a cross-platform and cross-programming model array of results of achievable memory bandwidth.

Performance and Power Efficient Massive Parallel Computational Model for HPC Heterogeneous Exascale Systems

The emerging high-performance computing Exascale supercomputing system, which is anticipated to be available in 2020, will unravel many scientific mysteries. This extraordinary processing framework will accomplish a thousand-folds increment in figuring power contrasted with the current Petascale framework. The prospective framework will help development communities and researchers in exploring from conventional homogeneous to the heterogeneous frameworks that will be joined into energy efficient GPU devices along with traditional CPUs. For accomplishing ExaFlops execution through the Ultrascale framework, the present innovations are confronting several challenges. Huge parallelism is one of these challenges, which requires a novel low power consuming parallel programming approach for attaining massive performance. This paper introduced a new parallel programming model that achieves massive parallelism by combining coarse-grained and fine-grained parallelism over inter-node and intra-node computation respectively. The proposed framework is tri-hybrid of MPI, OpenMP, and compute unified device architecture (MOC) that compute input data over heterogeneous framework. We implemented the proposed model in linear algebraic dense matrix multiplication application, and compared the quantified metrics with well-known basic linear algebra subroutine libraries such as CUDA basic linear algebra subroutines library and KAUST basic linear algebra subprograms. MOC outperformed to all implemented methods and achieved massive performance by consuming less power. The proposed MOC approach can be considered an initial and leading model to deal emerging Exascale computing systems.

Secure and Privacy Preserving Mail Servers using Modified Homomorphic Encryption (MHE) Scheme

Electronic mail (Email) or the paperless mail is becoming the most acceptable, faster and cheapest way of formal and informal information sharing between users. Around 500 billion mails are sent each day and the count is expected to be increasing. Today, even the sensitive and private information are shared through emails, thus making it the primary target for attackers and hackers. Also, the companies having their own mail server, relies on cloud system for storing the mails at a lower cost and maintenance. This affected the privacy of users as the searching pattern is visible to the cloud. To rectify this, we need to have a secure architecture for storing the emails and retrieve them according to the user queries. Data as well as the queries and computations to retrieve the relevant mails should be hidden from the third party. This article proposes a modified homomorphic encryption (MHE) technique to secure the mails. Homomorphic encryption is made practical using MHE and by incorporating Map Reduce parallel programming model, the execution time is exponentially reduced. Well known techniques in information retrieval, like Vector Space model and Term Frequency – Inverse Document Frequency (TF-IDF) concepts are utilized for finding relevant mails to the query. The analysis done on the dataset proves that our method is efficient in terms of execution time and in ensuring the security of the data and the privacy of the users.