An Efficient Dynamic Job Ordering and Slot Configuration for Minimizing the MakespanOf MapReduce Jobs

Abstract

MapReduce is a well-known parallel computing paradigm for massive data processing in clusters and data facilities.It is observed that different job execution orders and MapReduce slot configurations for a MapReduce workload have the enormously extraordinary efficiency related to the makespan, complete completion time, process utilization and otherperformance metrics. This paper proposes two classes of algorithms to diminish the lifespan and the total completion time for an offline MapReduce workload. Our first class of algorithms focuses on the job ordering optimization for a MapReduceworkload below a given map/reduce slot configuration. In distinction, our 2nd type of algorithms considers the scenariothat we will participate in optimization for map/reduce down slot configuration for a MapReduce workload.