Improving Data Locality and Avoiding Job Starvation by Implementing a Novel Scheduling Scheme

Abstract

In the MapReduce, most of the scheduling algorithms are designed to gain the node locality along with rack locality for traditional MapReduce clusters, in preference to reaching the VPS-locality as well Cen locality for virtual MapReduce clusters. At present, developing as well maintaining the traditional MapReduce cluster is expensive for big data users with low budget. Hence, we need to improve the data locality and implement the cost-effective virtual clusters. In this paper, we are implementing an efficient scheduling scheme named as hybridjob-driven scheduling scheme (JoSS). By implementing this scheme, we can achieve the data locality, avoid the job starvation and we can improve the Virtual cluster performance.