Volunteer Computing Environment Research Articles

IoT-EMS: An Internet of Things Based Environment Monitoring System in Volunteer Computing Environment

Environment monitoring is an important area apart from environmental safety and pollution control. Such monitoring performed by the physical models of the atmosphere is unstable and inaccurate. Machine Learning (ML) techniques on the other hand are more robust in capturing the dynamics in the environment. In this paper, a novel approach is proposed to build a cost-effective standardized environment monitoring system (IoT-EMS) in volunteer computing environment. In volunteer computing, the volunteers (people) share their resources for distributed computing to perform a task (environment monitoring). The system is based on the Internet of Things and is controlled and accessed remotely through the Arduino platform (volunteer resource). In this system, the volunteers record the environment information from the surrounding through different sensors. Then the sensor readings are uploaded directly to a web server database, from where they can be viewed anytime and anywhere through a website. Analytics on the gathered time-series data is achieved through ML data modeling using R Language and RStudio IDE. Experimental results show that the system is able to accurately predict the trends in temperature, humidity, carbon monoxide level, and carbon dioxide. The prediction accuracy of different ML techniques such as MLP, k-NN, multiple regression, and SVM are also compared in different scenarios.

The impact factors on the competence of big data processing

A major challenge of using volunteer computing (VC) for big data problems is the opportunistic environment. In such a dynamic, unreliable, and heterogeneous environment, there exist many uncertain factors that may impair the competence of big data processing. This paper explores these factors and their impact, aiming at exposing the original impaired performance that the undedicated environment of VC can achieve under the impact. Our investigation on this issue is four-fold. First, we define a number of impact factors to represent the opportunistic features of VC environment. Second, we model a distributed hash table-based MapReduce approach to process big data. Third, we proposes an ideal computing environment and then inject impact factors into the running MapReduce to quantitatively evaluate how the performance is impaired. Finally, we analyze the evaluation results to figure out the cause of impact and predict optimization potentials. The developers, who plan to construct MapReduce frameworks by using commodity computers or voluntary cycles on the public internet, will benefit from the evaluation results when considering performance requirements.

Data Processing Model to Perform Big Data Analytics in Hybrid Infrastructures

Big Data applications are present in many areas such as financial markets, search engines, stream services, health care, social networks, and so on. Data analysis provides value to information for organizations. Classical Cloud Computing represents a robust architecture to perform complex and large-scale computing for these areas. The main challenges are the user's unknowledge about Cloud infrastructure, the requirement needed for improving performance, and the resource management to maintain stable processing. In these difficulties, an inadequate solution can lead to users overestimate or underestimate the number of computational resources, which drives to the budget increases. One way to work around this problem is to make use of Volunteer Computing since it provides distributed computational resources at free monetary cost. However, a volatile machine behavior is a problem to address in Big Data data distributions. Thus, this work proposes a data distribution model composed of Cloud Computing and Volunteer Computing environments in a hybrid fashion for Big Data analytics. The contributions of this work are: i) the required evaluation to enable efficient deployment of Big Data in hybrid infrastructures; ii) the development of an HR_Alloc Algorithm for establishing the data placement to Big Data applications; iii) a model to resource allocation in hybrid infrastructures. The obtained results indicate the feasibility of using a hybrid infrastructure with up to 35% of unstable machines in the worst-case scenario, without losing performance and a monetary cost lower than 20% in comparison to Classical Cloud Computing. Also, communication costs decrease up to 57.14% in the best-case scenario due to load balancing.

A Robust and Efficient Message Passing Library for Volunteer Computing Environments

The objective of this research is to convert ordinary idle PCs into virtual clusters for executing parallel applications. The paper presents VolpexMPI that is designed to enable seamless forward application progress in the presence of frequent node failures as well as dynamically changing networks and node execution speeds. Process replication is employed to provide robustness. The central challenge in the design of VolpexMPI is to efficiently and automatically manage dynamically varying number of process replicas in different states of execution progress. The key fault tolerance technique employed is fully distributed sender based logging. The paper presents the design and an implementation of VolpexMPI. Preliminary results validate that the overhead of providing robustness is modest for applications with a favorable ratio of communication to computation and a low degree of communication.

An Adaptive Priority Tuning System for Optimized Local CPU Scheduling using BOINC Clients

Volunteer Computing (VC) is a Distributed Computing model which utilizes idle CPU cycles from computing resources donated by volunteers who are connected through the Internet to form a very large-scale, loosely coupled High Performance Computing environment. Distributed Volunteer Computing environments such as the BOINC framework is concerned mainly with the efficient scheduling of the available resources to the applications which require them. The BOINC framework thus contains a number of scheduling policies/algorithms both on the server-side and on the client which work together to maximize the available resources and to provide a degree of QoS in an environment which is highly volatile. This paper focuses on the BOINC client and introduces an adaptive priority tuning client side middleware application which improves the execution times of Work Units (WUs) while maintaining an acceptable Maximum Response Time (MRT) for the end user. We have conducted extensive experimentation of the proposed system and the results show clear speedup of BOINC applications using our optimized middleware as opposed to running using the original BOINC client.

Generalized Spot-Checking for Reliable Volunteer Computing

While volunteer computing (VC) systems reach the most powerful computing platforms, they still have the problem of guaranteeing computational correctness, due to the inherent unreliability of volunteer participants. Spot-checking technique, which checks each participant by allocating spotter jobs, is a promising approach to the validation of computation results. The current spot-checking is based on the implicit assumption that participants never distinguish spotter jobs from normal ones; however generating such spotter jobs is still an open problem. Hence, in the real VC environment where the implicit assumption does not always hold, spot-checking-based methods such as well-known credibility-based voting become almost impossible to guarantee the computational correctness. In this paper, we generalize spot-checking by introducing the idea of imperfect checking. This generalization allows to guarantee the computational correctness under the situation that spot-checking is not fully-reliable and participants may distinguish spotter jobs. Moreover, we develop a generalized formula of the credibility, which enables credibility-based voting to utilize check-by-voting technique. Simulation results show that check-by-voting improves the performance of credibility-based voting, while guaranteeing the same level of computational correctness.

Computing Low Latency Batches with Unreliable Workers in Volunteer Computing Environments

Internet based volunteer computing projects such as SETI@home are currently restricted to performing coarse grained, embarrassingly parallel master-worker style tasks. This is partly due to the “pull” nature of task distribution in volunteer computing environments, where workers request tasks from the master rather than the master assigning tasks to arbitrary workers. In this paper we propose algorithms for computing batches of medium grained tasks with deadlines in pull-style volunteer computing environments. We develop models of unreliable workers based on analysis of trace data from an actual volunteer computing project. These models are used to develop algorithms for task distribution in volunteer computing systems with a high probability of meeting batch deadlines. We develop algorithms for perfectly reliable workers, computation-reliable workers and unreliable workers. Finally, we demonstrate the effectiveness of the algorithms through simulations using traces from actual volunteer computing environments.

Construction of a Cooperative-Server-Group-Based Volunteer Computing Environment

Construction of a Cooperative-Server-Group-Based Volunteer Computing Environment