Desktop Grid Research Articles

Federated Deep Learning for Solving an Image Classification Problem on a Desktop Grid System

Federated Deep Learning for Solving an Image Classification Problem on a Desktop Grid System

WukaStore: Scalable, Configurable and Reliable Data Storage on Hybrid Volunteered Cloud and Desktop Systems

In this paper, we propose a hybrid storage framework WukaStore, which offers a scalable, configurable and reliable big data storage service by integrating stable storage (such as Cloud storage or durable storage utilities) and volatile storage (such as idle storage harnessed from desktop PCs over the Internet). By configuring different storage strategies, WukaStore delivers cost-effective storage service to satisfy users’ requirements. We present trace-driven simulations to study the impact of different strategies and how to ensure high availability and durability in WukaStore. We present the prototype implementation of WukaStore based on BitDew, an open source middleware for Big Data management. We conduct performance evaluations to validate the effectiveness of WukaStore, through the deployment of WukaStore in the French Grid'5000 experimental platform. In particular, we also present a case study where WukaStore is deployed in a hybrid environment taking advantage of storage provided by Amazon S3, Dropbox and a local Desktop Grid at the same time. Our evaluation results show that WukaStore provides users with an alternative method to store their big data on hybrid volunteered cloud and desktop systems and to decrease the cost of data storage, while providing great data access performance, scalability as well as reliability.

Metrics of Efficiency and Performance when Using Evolutionary Algorithm on Desktop Grids

Metrics of Efficiency and Performance when Using Evolutionary Algorithm on Desktop Grids

Использование специализированных вычислителей на узле грид-системы для решения комбинаторных задач.

The paper discusses the use of specialized Intel Xeon Phi devices in a desktop grid system. As an example of the successful use of specialized devices, the combinatorial problem of finding diagonal Latin squares is given. Features of implementation of the computing application are discussed. The results of work of a computer application in multithreaded modes on different processors andspecialized devices are given.

Зонтичный проект распределенных вычислений E-governance.

The paper deals with desktop grid systems based on the BOINC platform. Various ways of increasing the efficiency of computations on grid systems are discussed. The umbrella distributed computing project is considered as a convenient tool for conducting various computational experiments. The advantages of the umbrella project ofdistributed computing and the measures for improving the functioning of the umbrella project are described. The test experiments conducted on the E-governance project are listed.

Dynamical systems analysis using many-task interactive cloud computing

The article states an approach for developing a distributed application architecture for dynamic systems analysis in a hybrid cloud. The problem that was posed as an example is based on the calculation of the Lyapunov characteristic exponents for trajectories of dynamical systems in phase space. Parallelization of the computational process is achieved by supplying the list of running tasks for computing Lyapunov characteristic exponents, depending on the result of finished tasks during calculations. We have implemented a distributed application to make this experiment automated. The architecture of application elements is established on the principles of the enterprise desktop grid and cloud systems. A vital detail of the application architecture is the deployment process based on a cloud hybrid computing environment using the Jupyter server running on the DigitalOcean cloud platform and the Everest cloud platform simultaneously with the Maple package.

A Hybrid Batch Mode Fault Tolerance Strategy in Desktop Grids

Desktop grids make use of unused resources of personal computers provided by volunteers to work as a huge processor and make them available to users that need them. The rate of heterogeneity, volatility, and unreliability is higher in case of a desktop grid in comparison to conventional systems. Therefore, the application of fault tolerance strategies becomes an inevitable requirement. In this article, a hybrid fault tolerance strategy is proposed which works in three phases. First, two phases deal with the task and resource scheduling in which appropriate scheduling decisions are taken in order to select the most suitable resource for a task. Even if any failure occurs, it is then recovered in the third phase by using rescheduling and checkpointing. The proposed strategy is compared against existing hybrid fault tolerance scheduling strategies and ensures a 100% success rate and processor utilization and outperforms by a factor of 3.5%, 0.4%, and 0.1% when turnaround time, throughput, and makespan, respectively, are taken into account

An Adaptive Fault Tolerant Scheduling System for Desktop Grid

In Desktop Grid, managing faults is very crucial and challenging job. So a fault tolerant system is mandatory requirement in desktop grid for fault identification and their resolution. A fault tolerant system allows applications to continue execution despite having faults without termination. In this paper,an adaptive fault tolerant scheduling system is presented that combines dynamic replication with rescheduling. The system initially schedules jobs depending upon the completion time and fault rate of resources and then fault-tolerant strategies are applied depending upon availability of resources. To measure the performance,experiments has been carried out and it has been observed that proposed scheduling system outperforms by a factor of 4.8% w.r.t. Average task response time and 0.02% w.r.t. Average flowtime as compared to existing system. On the other hand, there is no significant improvement is observed when BoT completion time and average execution time are compared to existing system.

Game-theoretic model of a volunteer computer GRID

In the paper we propose a simple game-theoretic model of a Desktop Grid for volunteer computing. Task replication reduces the risk of accepting wrong answers due to sabotage. Saboteur's attack by intruding multiple computing nodes brings him some profit in case a wrong answer is accepted, while the server suffers some penalty in this case. Nodes are assigned some reputation as a monotone function of the number of produced correct (or not exposed) answers. We obtain the optimal mixed strategies and show that the average gain of the players depends only on the server's penalty, nodes' reputation, and the size of the subgrid of nodes with the same reputation. Also we estimate the server's cost per an answer. Numerical examples show that the average cost of the server is not more than that in the case when the number of intruders is known.

Динамическое построение прогноза времени завершения вычислительного эксперимента в Desktop Grid

Desktop Grid is a powerful tool to perform high-throughput computing. Desktop Grid is a form of distributed high-throughput computing system, which uses idle time of non-dedicated geographically distributed computing nodes connected over low-speed network. It has significant differences from computing clusters and computational GRIDs, and needs special operation tackle. In this paper, we present a mechanism for dynamic forecasting of the completion time of a computational experiment in a Desktop Grid. We propose a statistical approach based on the linear regression model with the calculation of a confidence interval, taking into account the accumulation of statistical error and, if needed, changing the forecast. The developed approach is used to implement a forecasting algorithm and software module for a Desktop Grid. We present experimental results based on data from the RakeSearch volunteer computing project.

A Survey of Desktop Grid Scheduling

The paper surveys the state of the art of task scheduling in Desktop Grid computing systems. We describe the general architecture of a Desktop Grid system and the computing model adopted by the BOINC middleware. We summarize research papers to bring together and examine the optimization criteria and methods proposed by researchers so far for improving Desktop Grid task scheduling (assigning tasks to computing nodes by a server). In addition, we review related papers, which address non-regular architectures, like hierarchical and peer-to-peer Desktop Grids, as well as Desktop Grids designed for solving interdependent tasks.

Effective scanning of parameter space in a Desktop Grid for identification of a hydride decomposition model

We consider parameter identification of a hydride decomposition model by scanning the parameter space in parallel. Such problem is resource demanding, but suits best for Desktop Grid computing. Considering task retrieval as a game, we show that the search process can be improved to produce solutions faster in comparison with random search, with no or minor additional cost.

Using virtualization to protect the proprietary material science applications in volunteer computing

Abstract USPEX is a world-leading software for computational material design. In essence, USPEX splits simulation into a large number of workunits that can be processed independently. This scheme ideally fits the desktop grid architecture. Workunit processing is done by a simulation package aimed at energy minimization. Many of such packages are proprietary and should be protected from unauthorized access when running on a volunteer PC. In this paper we present an original approach based on virtualization. In a nutshell, the proprietary code and input files are stored in an encrypted folder and run inside a virtual machine image that is also password protected. The paper describes this approach in detail and discusses its application in USPEX@home volunteer project.

Efficient Redundancy Techniques in Cloud and Desktop Grid Systems using MAP/G/c-type Queues

Abstract Cloud computing is continuing to prove its flexibility and versatility in helping industries and businesses as well as academia as a way of providing needed computing capacity. As an important alternative to cloud computing, desktop grids allow to utilize the idle computer resources of an enterprise/community by means of distributed computing system, providing a more secure and controllable environment with lower operational expenses. Further, both cloud computing and desktop grids are meant to optimize limited resources and at the same time to decrease the expected latency for users. The crucial parameter for optimization both in cloud computing and in desktop grids is the level of redundancy (replication) for service requests/workunits. In this paper we study the optimal replication policies by considering three variations of Fork-Join systems in the context of a multi-server queueing system with a versatile point process for the arrivals. For services we consider phase type distributions as well as shifted exponential and Weibull. We use both analytical and simulation approach in our analysis and report some interesting qualitative results.

Improving “tail” computations in a BOINC-based Desktop Grid

AbstractA regular Desktop Grid bag-of-tasks project can take a lot of time to complete computations. An important part of the process is tail computations: when the number of tasks to perform becomes less than the number of computing nodes. At this stage, a dynamic replication could be used to reduce the time needed to complete computations. In this paper, we propose a mathematical model and a strategy of dynamic replication at the tail stage. The results of the numerical experiments are given.

Task Scheduling in Desktop Grids: Open Problems

AbstractWe survey the areas of Desktop Grid task scheduling that seem to be insufficiently studied so far and are promising for efficiency, reliability, and quality of Desktop Grid computing. These topics include optimal task grouping, “needle in a haystack” paradigm, game-theoretical scheduling, domain-imposed approaches, special optimization of the final stage of the batch computation, and Enterprise Desktop Grids.

Video conference in the fog: an economical approach based on enterprise desktop grid

There exist two classical and well-understood approaches to video-processing tasks (such as mixing or trans-coding) for videoconferencing. The first one is using a centralized multipoint control unit (MCU), hardware- or software-based, deployed on-premises or in the cloud. In the second approach, the video-processing tasks are directly handled in endpoints (i.e., equipment such as PCs, laptops, and tablets that are involved in the video session). Performance is then restricted by device characteristics, especially in the case of mobile devices. In this paper, we propose a third alternative approach. It has been shown that there exist significant computational resources in user equipment deployed in enterprises, which are under-utilized most of the time. In this paper, we propose a system, which distributes real-time video-processing tasks on these available resources. A dedicated multi-attribute decision-making (MADM) method is designed in order to take into account the variety of attributes impacting Quality of Experience. We enumerate a comprehensive list of events, which may cause distribution or redistribution of video-processing activities and provide simple algorithms to tackle all these cases. We then test the MADM algorithm by means of simulations in order to study the impact of the main critical parameters.

Computing Extremely Large Values of the Riemann Zeta Function

The paper summarizes the computation results pursuing peak values of the Riemann zeta function. The computing method is based on the RS-Peak algorithm by which we are able to solve simultaneous Diophantine approximation problems efficiently. The computation environment was served by the SZTAKI Desktop Grid operated by the Laboratory of Parallel and Distributed Systems at the Hungarian Academy of Sciences and the ATLAS supercomputing cluster of the Eotvos Lorand University, Budapest. We present the largest Riemann zeta value known till the end of 2016.

Trust model for reliable node allocation based on daily computer usage behavior

SummaryResource volatility is a major challenge on desktop grid platforms with characteristics that primarily depend on human computer usage behavior. This study proposes a trust model based on Dempster–Shafer theory that predicts the relative reliability of nodes using information on daily computer usage behavior based on the historical information from a desktop grid platform for a desktop grid. In the proposed trust model, called TM‐DG, a degree of trustworthiness is introduced for the nodes to quantify their reliability. Dempster's rule of combination is also applied to exploit two bodies of independent evidence: 1) current node availability as actively probed by a special test node and 2) proportion of allocated tasks successfully completed. The simulations performed on a lightweight desktop grid platform illustrate how TM‐DG can fully utilize the most reliable nodes for a given computation, leading to a reduction in the communication overhead and an improvement in the computing power of the platform.

ComBos: A complete simulator of Volunteer Computing and Desktop Grids

Volunteer Computing is a type of distributed computing in which ordinary people donate their idle computer time to science projects like SETI@Home, Climateprediction.net and many others. In a similar way, Desktop Grid Computing is a form of distributed computing in which an organization uses its existing computers to handle its own long-running computational tasks. BOINC is the main middleware that provides a software platform for Volunteer Computing and desktop grid computing, and it became generalized as a platform for distributed applications in areas as diverse as mathematics, medicine, molecular biology, climatology, environmental science, and astrophysics. In this paper we present a complete simulator of BOINC infrastructures, called ComBoS. Although there are other BOINC simulators, none of them allow us to simulate the complete infrastructure of BOINC. Our goal was to create a complete simulator that, unlike the existing ones, could simulate realistic scenarios taking into account the whole BOINC infrastructure, that other simulators do not consider: projects, servers, network, redundant computing, scheduling, and volunteer nodes. The outputs of the simulations allow us to analyze a wide range of statistical results, such as the throughput of each project, the number of jobs executed by the clients, the total credit granted and the average occupation of the BOINC servers. The paper describes the design of ComBoS and the results of the validation performed. This validation compares the results obtained in ComBoS with the real ones of three different BOINC projects (Einstein@Home, SETI@Home and LHC@Home). Besides, we analyze the performance of the simulator in terms of memory usage and execution time. The paper also shows that our simulator can guide the design of BOINC projects, describing some case studies using ComBoS that could help designers verify the feasibility of BOINC projects.