Abstract
Privacy protection is an important problem in workflow scheduling, for which scheduling tasks with privacy constraints to reliable resources is essential. In this article, we consider the scheduling problem of Spark applications in a hybrid cloud with deadline and privacy constraints. A scheduling algorithm framework is proposed, which consists of four algorithm components. Candidate strategies are developed for each algorithm component. The components and parameter values are statistically calibrated over a comprehensive set of random instances. The proposed algorithm is compared to modified classical algorithms for similar problems. The experimental results indicate that the proposed algorithm outperforms the compared algorithms under different application scales, deadlines, and private VMs.
Highlights
A hybrid cloud is the combination of private and public clouds
Only homogeneous computing resources were considered in the traditional Spark framework, while resources are heterogeneous in practice
All the tested algorithms are coded in Java and run on an Intel Core i5-3230 CPU @ 2.60 GHZ with 8 GB of RAM
Summary
A hybrid cloud is the combination of private and public clouds It has become a popular resource provision choice for an increasing number of enterprises for processing big data tasks because both resource utilization and scalability are considered. We consider the problem of scheduling a Spark application with a deadline and data privacy to a hybrid cloud to minimize the total rental cost. W. Hu et al.: Hybrid Cloud Workflow Scheduling Method With Privacy Data considered problem. Based on the classical Spark data processing framework, a modified Spark scheduling architecture is constructed to meet the task privacy constraint with heterogeneous resources in a hybrid cloud.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
