Master-worker Model Research Articles

A Web-Based Docker Image Assistant Generation Tool for User-PC Computing System

Currently, we are developing the user-PC computing (UPC) system based on the master-worker model as a scalable, low-cost, and high-performance computing platform. To run various application programs on personal computers (PCs) with different environments for workers, it adopts Docker technology to bundle every necessary software as one image file. Unfortunately, the Docker file/image are manually generated through multiple steps by a user, which can be the bottleneck. In this paper, we present a web-based Docker image assistant generation (DIAG) tool in the UPC system to assist or reduce these process steps. It adopts Angular JavaScript for offering user interfaces, PHP Laravel for handling logic using RestAPI, MySQL database for storing data, and Shell scripting for speedily running the whole program. In addition, the worker-side code modification function is implemented so that a user can modify the source code of the running job and update the Docker image at a worker to speed up them. For evaluations, we collected 30 Docker files and 10 OpenFOAM jobs through reverse processing from Docker images in Github and generated the Docker images using the tool. Moreover, we modified source codes for network simulations and generated the Docker images in a worker five times. The results confirmed the validity of the proposal.

Worker Assignment for Multiple Masters to Speed up Coded Distributed Computing in Heterogeneous Clusters

In distributed computing systems, coding has played an important role to robustify the system against the effect of noise, e.g., stragglers, system failures and communication bottlenecks. Most of the existing work has focused on a simple master-worker model with one master and homogeneous workers. However, real-world systems are typically configured with heterogeneous workers distributed to computing nodes and serve multiple tasks in parallel. In this study, we consider the scenario in which multiple masters perform matrix multiplications using the workers having group heterogeneity. The group heterogeneity models that homogeneous workers are located in the same location and regarded as a group; the workers deployed in the different locations are potentially heterogeneous. We propose an asymptotically optimal worker assignment to multiple masters for coded distributed computing in the presence of heterogeneous groups of workers. Specifically, we present a lower bound for the expected latency in terms of the numbers of workers assigned to the masters and the amount of tasks allocated to workers. Adding the concentration constraints on the number of workers allocated to masters, we can obtain the minimum of the lower bound by taking the optimal worker assignment. We find the optimal worker assignment by converting the problem at hand into a linear programming problem. From both numerical simulations and experiments on Amazon EC2 clusters, we confirm that the effect of the proposed worker assignment is significant in various scenarios.

A parallel memetic algorithm with explicit management of diversity for the job shop scheduling problem

The job shop scheduling problem (JSSP) is a very popular NP-hard optimization problem that involves assigning jobs to resources. Recent advances in the field of memetic algorithms show that explicitly managing the diversity of the population by taking into account the stopping criterion with the aim of dynamically adapting the balance between exploration and exploitation is key to their success. This is especially the case in long-term executions. However, this design principle has not yet been applied to the JSSP. This paper proposes a novel memetic algorithm that integrates some of the most advanced components devised in the literature for the JSSP with a replacement strategy that explicitly manages the diversity by considering a novel dissimilarity measure. To properly address large instances, a parallel master-worker model is used. Experimental validation shows the important advances attained by our proposal when compared to two state-of-the-art optimizers. The advantages are clear in both sequential and parallel cases, with more impressive achievements appearing in the parallel case. The parallel proposal has yielded new best-known solutions in 30 well-known JSSP instances, matching the lower bound in two of them, meaning that at least two new optimal solutions have been discovered.

Parallelizing Subgradient Methods for the Lagrangian Dual in Stochastic Mixed-Integer Programming

The dual decomposition of stochastic mixed-integer programs can be solved by the projected subgradient algorithm. We show how to make this algorithm more amenable to parallelization in a master-worker model by describing two approaches, which can be combined in a natural way. The first approach partitions the scenarios into batches and makes separate use of subgradient information for each batch. The second approach drops the requirement that evaluation of function and subgradient information is synchronized across the scenarios. We provide convergence analysis of both methods. We also evaluate their performance on two families of problems from SIPLIB on a single server with 32 single-core worker processes, demonstrating that when the number of workers is high relative to the number of scenarios, these two approaches (and their synthesis) can significantly reduce running time.

Research on the Task Assignment Problem with Maximum Benefits in Volunteer Computing Platforms

As a type of distributed computing, volunteer computing (VC) has provided unlimited computing capacity at a low cost in recent decades. The architecture of most volunteer computing platforms (VCPs) is a master–worker model, which defines a master–slave relationship. Therefore, VCPs can be considered asymmetric multiprocessing systems (AMSs). As AMSs, VCPs are very promising for providing computing services for users. Users can submit tasks with deadline constraints to the VCPs. If the tasks are completed within their deadlines, VCPs will obtain the benefits. For this application scenario, this paper proposes a new task assignment problem with the maximum benefits in VCPs for the first time. To address the problem, we first proposed a list-based task assignment (LTA) strategy, and we proved that the LTA strategy could complete the task with a deadline constraint as soon as possible. Then, based on the LTA strategy, we proposed a maximum benefit scheduling (MBS) algorithm, which aimed at maximizing the benefits of VCPs. The MBS algorithm determined the acceptable tasks using a pruning strategy. Finally, the experiment results show that our proposed algorithm is more effective than current algorithms in the aspects of benefits, task acceptance rate and task completion rate.

Seeding and adjoining zero-halo partitioned parallel scientific codes

ABSTRACTAlgorithmic differentiation tools can automate the adjoint transformation of parallel message-passing codes [J. Utke, L. Hascoët, P. Heimbach, C. Hill, P. Hovland, and U. Naumann, Toward Adjoinable MPI, in 2009 IEEE International Symposium on Parallel & Distributed Processing, May, IEEE, 2009, pp. 1–8] using the AMPI library. Nevertheless, a non-trivial and manual step after the differentiation is the initialization of the seed and retrieval of the output values from the differentiated code. Ambiguities in seeding occur in programs where the user is unable to expose the complete program flow with a single entry and single exit point to the AD tool. We present the ambiguities associated with seed initialization and output retrieval for adjoint transformation of halo and zero-halo partitioned MPI programs. We introduce a general framework to eliminate ambiguities in seeding and retrieval for shared-node reduction over +, and * operators using a conceptual master-worker model. The model shows the need for new MPI calls for retrieval and eliminate MPI calls for seed initialization. Different implementations for seeding manually assembled adjoints were inferred from the model, namely, partial and unique seeding. We successfully applied the seeding techniques to a 3D zero-halo partitioned unstructured compressible discrete adjoint solver and highlight the merits and demerits of each strategy.

The Influence of Memory-Aware Computation on Distributed BLAST

Background: One of the pivotal challenges in nowadays genomic research domain is the fast processing of voluminous data such as the ones engendered by high-throughput Next-Generation Sequencing technologies. On the other hand, BLAST (Basic Local Alignment Search Tool), a longestablished and renowned tool in Bioinformatics, has shown to be incredibly slow in this regard. Objective: To improve the performance of BLAST in the processing of voluminous data, we have applied a novel memory-aware technique to BLAST for faster parallel processing of voluminous data. Method: We have used a master-worker model for the processing of voluminous data alongside a memory-aware technique in which the master partitions the whole data in equal chunks, one chunk for each worker, and consequently each worker further splits and formats its allocated data chunk according to the size of its memory. Each worker searches every split data one-by-one through a list of queries. Results: We have chosen a list of queries with different lengths to run insensitive searches in a huge database called UniProtKB/TrEMBL. Our experiments show 20 percent improvement in performance when workers used our proposed memory-aware technique compared to when they were not memory aware. Comparatively, experiments show even higher performance improvement, approximately 50 percent, when we applied our memory-aware technique to mpiBLAST. Conclusion: We have shown that memory-awareness in formatting bulky database, when running BLAST, can improve performance significantly, while preventing unexpected crashes in low-memory environments. Even though distributed computing attempts to mitigate search time by partitioning and distributing database portions, our memory-aware technique alleviates negative effects of page-faults on performance.

General-purpose coordinator–master–worker model for efficient large-scale simulation over heterogeneous infrastructure

In this work, we propose a general-purpose coordinator–master–worker (GP-CMW) model to enable efficient and scalable simulation. The model supports distributed and parallel simulation over a heterogeneous computing node architecture with both multi-core CPUs and GPUs. The model aims at maximizing the hardware activity rate while reducing the overall management overhead. The proposed model includes five components: coordinator, priority abstraction layer, master, hardware abstraction layer, and worker. The proposed model is mainly optimized for large-scale simulation that relies on massive parallelizable events. Extensive set of experimental results shows that GP-CMW provides a significant gain from medium to intensive simulation load by exploiting heterogeneous computing resources including CPU and GPU. Regarding simulation runtime, the proposed GP-CMW model delivers a speedup that is 3.6 times faster than the CMW model.

A User-PC Computing System as Ultralow-cost Computation Platform for Small Groups

Recently, a personal computer (PC) has rapidly evolved the computing ability due to the highest CPU clock, more CPU cores, and the larger memory. Then, the collection of idling CPU cycles in conventional PCs of the members in a group such as a university laboratory significantly enhances the level of parallel computing platform with very small costs. This low-cost computation environment is of great interest in schools, especially in developing countries. To realize this concept, in this paper, we propose a User-PC Computing (UPC) system based on the master-worker model. First, we present the design and implementation of the UPC master using Linux and for the UPC worker with both Linux and Windows. Then, because the job scheduling takes a key role in determining the overall computing performance, we present an efficient job scheduling method using performance measurement results of worker PCs by benchmark programs, in addition to a conventional round-robin one. We verify the effectiveness of our proposed UPC system through experiments using several benchmark and application programs.

Multi-core versus many-core computing for many-task Branch-and-Bound applied to big optimization problems

On the road to exascale, coprocessors are increasingly becoming key building blocks of High Performance Computing platforms. In addition to their energy efficiency, these many-core devices boost the performance of multi-core processors. In this paper, we revisit the design and implementation of Branch-and-Bound (B&B) algorithms for multi-core processors and Intel Xeon Phi coprocessors considering the offload mode as well as the native one. In addition, two major parallel models are considered: the master–worker and the work pool models. We address several parallel computing issues including processor–coprocessor data transfer optimization and vectorization. The proposed approaches have been experimented using the Flow-Shop scheduling problem (FSP) and two hardware configurations equivalent in terms of energy consumption: Intel Xeon E5-2670 processor and Intel Xeon Phi 5110P coprocessor. The reported results show that: (1) the proposed vectorization mechanism reduces the execution time by 55.4% (resp. 30.1%) in the many-core (resp. multi-core) approach ; (2) the offload mode allows a faster execution on MIC than the native mode for most FSP problem instances ; (3) the many-core approach (offload or native) is in average twice faster than the multi-core approach ; (4) the work pool parallel model is more suited for many/multi-core B&B applied to FSP than the master–worker model because of its irregular nature.

Robust and portable capacity computing method for many finite element analyses of a high-fidelity crustal structure model aimed for coseismic slip estimation

Computation of many Green's functions (GFs) in finite element (FE) analyses of crustal deformation is an essential technique in inverse analyses of coseismic slip estimations. In particular, analysis based on a high-resolution FE model (high-fidelity model) is expected to contribute to the construction of a community standard FE model and benchmark solution. Here, we propose a naive but robust and portable capacity computing method to compute many GFs using a high-fidelity model, assuming that various types of PC clusters are used. The method is based on the master–worker model, implemented using the Message Passing Interface (MPI), to perform robust and efficient input/output operations. The method was applied to numerical experiments of coseismic slip estimation in the Tohoku region of Japan; comparison of the estimated results with those generated using lower-fidelity models revealed the benefits of using a high-fidelity FE model in coseismic slip distribution estimation. Additionally, the proposed method computes several hundred GFs more robustly and efficiently than methods without the master–worker model and MPI.

Dynamic MPI parallel task scheduling based on a master-worker pattern in cloud computing

Load imbalance issues have become one of the main challenges in efficient task scheduling. On-demand computing resources that can be provided by the cloud infrastructure enable cost-efficiency for numerous application cases compared to on-premise resources that an organisation purchases and that might idle for non-peak situations. However, scheduling a large amount of tasks in parallel on the cloud nodes cannot always maintain the promised cost-efficiency due to the different workloads arising on these cloud nodes, caused by single point of failure, low bandwidth, and other unforeseen situations. Generated overhead and load imbalances between nodes lead to numerous paid resources lay idle. In our work, we propose a dynamic parallel task scheduling method by employing a master-worker model on a real-world engineering application executed on the Azure cloud. The main idea of our work is that we schedule tasks on cloud compute resources depending on the actual workload of each process instead of static-scheduled load.

A parallel algorithm with enhancements via partial objective value cuts for cluster-based wireless sensor network design

In this paper, we develop a parallel algorithm for the solution of an integrated topology control and routing problem in Wireless Sensor Networks (WSNs). After presenting a mixed-integer linear optimization formulation for the problem, for its solution, we develop an effective parallel algorithm in a Master–Worker model that incorporates three parallelization strategies, namely low-level parallelism, domain decomposition, and multiple search (both cooperative and independent) in a single Master–Worker framework.For improved algorithmic efficiency, we introduce three reduced subproblems and devise partial objective value cuts from these reduced models. We utilize both the reduced models, for which we suggest efficient approaches for their solution, and the associated cuts in our parallel algorithm. We observe that the reduced models provide valuable information on the optimal design variables for the original model and we exploit this fact in our parallel algorithm. Our overall parallelization scheme utilizes exact optimization models and solutions as its components and allows cooperation among multiple worker processors via communication of partial solution and cut information. Computational study shows that our approach is very effective in addressing this complex problem. Parallel implementation not only achieves a speed-up of the computations, but also yields better solutions as it can explore the solution space more effectively.

Shortest-linkage-based parallel hierarchical clustering on main-belt moving objects of the solar system

Data clustering is an important data preparation process in many scientific analysis researches. In astronomy, although the distributed environments and modern observation techniques enable users to collect and access huge amounts of data, the corresponding clustering process may become very costly. One of the challenges is that the sequential clustering algorithms, that can be applied to cluster hundreds of thousand main-belt asteroids to reason about the origins of the main-belt asteroids, may not be used in the distributed environment directly. Therefore, this study focuses on the problem of parallelizing the traditional hierarchical agglomerative clustering algorithm using shortest-linkage. We propose a new parallel hierarchical agglomerative clustering algorithm based on the master–worker model. The master process divides the whole computation into several small tasks, and distributes the tasks to the worker processes for parallel processing. Then, the master process merges the results from the worker processes to form a hierarchical data structure. The proposed algorithm uses a pruning threshold to reduce the execution time and the storage requirement during the computation. It also supports fast incremental update that merges new data items into a constructed hierarchical tree in seconds, given a tree of about 550,000 data items. To evaluate the performance of our algorithm, this study has conducted several experiments using the MPCORB dataset and a dataset from the DVO database. The results confirm the efficiency of our proposed methodology. Compared with prior similar studies, the proposed algorithm is more flexible and practical in the problem of distributed hierarchical agglomerative clustering.

How to Use Google App Engine for Free Computing

Can the Google App Engine cloud service be used, free of charge, to execute parameter study problems? That question drove this research, which is founded on the App Engine's newly developed Task Queue API. The authors created a simple and extensible framework implementing the master-worker model to enable usage of the App Engine application servers as computational nodes. This article presents and discusses the results of the feasibility study, as well as compares the solution with EC2, Amazon's free cloud offering.

Grid-Enabled Optimization with GAMS

We describe a framework for modeling optimization problems for solution on a grid computer. The framework is easy to adapt to multiple grid engines and can seamlessly integrate evolving mechanisms from particular computing platforms. It facilitates the widely used master-worker model of computing and is shown to be flexible and powerful enough for a large variety of optimization applications. In particular, we summarize a number of new features of the GAMS modeling system that provide a lightweight, portable, and powerful framework for optimization on a grid. We provide downloadable examples of its use for embarrasingly parallel financial applications, decomposition of complementarity problems, and for solving very difficult mixed-integer programs to optimality. Computational results are provided for a number of different grid engines, including multicore machines, a pool of machines controlled by the Condor resource manager, and the grid engine from Sun Microsystems.

階層サーバ・クライアント型モデルによる非同期並列分散遺伝的アルゴリズム

The most popular researches about Parallel GAs are implemented as; Population is devided into some subpopulations, each subpopulation executes GA independently and some individuals are migrated in fixed intervals or fixed probability. On the other hand, Grid Computing has been noticed and a research that implements Parallel GA by using Master-Worker model on Grid Computing has been reported. However, on the huge search space problems, Parallel GA by using Master-Worker model needs a lot of worker to get better solution quality. If there are a lot of workers, the traffic loads to the master. In this paper, we propose Asynchronous Parallel Distributed GA by using Layered Server-Client model. This model is based on Elite Migration on Server-Client model we proposed before. In this model, an Elite Server manages some Subpopulation Clients, and a Master Server manages some Elite Servers. From this structure, the number of Subpopulation Clients that a Elite Server manages is able to be reduced and the traffic on an Elite Server is also able to be reduced. To evaluate our proposed model, we apply to some problems. As the results, we confirm that the fitness is as well as that of current methods and the traffic is less than that of current methods. We also confirm that the migration time is able to be reduced especially in large search space problems.

A Linda-based Hierarchical Master-Worker Model

A Linda-based Hierarchical Master-Worker Model

A Moment‐Based Method for Large‐Scale Generalized Eigenvalue Problems

AbstractIn this paper we consider a method for finding several eigenvalues and corresponding eigenvectors of large‐scale generalized eigenvalue problems. In this method, a small matrix pencil that has only the desired eigenvalues is derived using complex moments obtained via numerical integration. Since the process to derive the moments can be performed in parallel and we do not need to exchange data between processes, the presented method is suitable for master‐worker programming models. We have implemented and tested the proposed method in a grid RPC (remote procedure call) system. Numerical examples illustrate the properties of our approach. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Design and implementation of a windows-based parallel computing environment for large scale optimization

A parallel computing environment to support optimization of large-scale engineering systems is designed and implemented on Windows-based personal computer networks, using the master–worker model and the Parallel Virtual Machine (PVM). It is involved in decomposition of a large engineering system into a number of smaller subsystems optimized in parallel on worker nodes and coordination of subsystem optimization results on the master node. The environment consists of six functional modules, i.e. the master control, the optimization model generator, the optimizer, the data manager, the monitor, and the post processor. Object-oriented design of these modules is presented. The environment supports steps from the generation of optimization models to the solution and the visualization on networks of computers. User-friendly graphical interfaces make it easy to define the problem, and monitor and steer the optimization process. It has been verified by an example of a large space truss optimization.