Improving the performance for task assignment in distributed server systems by partitioning the large tasks

Abstract

Task Assignment in distributed server systems focuses on the policy that assigns the tasks reached these systems in order to improve the response time. These tasks, generally, have the property that there is a tiny fraction (about 3%) of the large tasks that makes half (50%) of the total load. However, this property creates additional problems: the large tasks make the load difficult to balance among the servers, and the small tasks will be delayed by the large ones when they are in the same queue. In this paper, we propose a new policy for the Web clusters that we call Partitioning Large Tasks (PLT) and which deals with these problems mostly under a high traffic demand and a high variability of task sizes. PLT partitions each large task into fragments and assigns them to be processed in a parallel way and completing at the same time to improve the mean response time, and separates the small tasks from the large tasks to avoid being delayed. Performance tests show a significantly improvement in performance of PLT over the existing task assignment policies.