Manuscript Number : CSEIT1172488
Dynamic Mapreduce for Job Workloads through Slot Configuration Technique
Authors(2) :-M. Priyanka, Dr. A. Subramanyam The MapReduce is an open source Hadoop framework implemented for processing and producing distributed large Terabyte data on large clusters. Its primary duty is to minimize the completion time of large sets of MapReduce jobs. Hadoop Cluster only has predefined fixed slot configuration for cluster lifetime. This fixed slot configuration may produce long completion time (Makespan) and low system resource utilization. The current open source Hadoop allows only static slot configuration, like fixed numbers of map slots and reduce slots throughout the cluster lifetime. Such static configuration may lead to long completion length as well as low system resource utilizations. Propose new schemes which use slot ratio between map and reduce tasks as a tunable knob for minimizing the completion length (i.e., makespan) of a given set. By leveraging the workload information of recently completed jobs, schemes dynamically allocates resources (or slots) to map and reduce tasks.. Many scheduling methodologies are discussed that aim to improve execution performance as well as completion time goal.
M. Priyanka Map Reduce, Makespan, Workload, Dynamic Slot Allocation. Publication Details Published in : Volume 2 | Issue 4 | July-August 2017 Article Preview
M.Tech., (PG Scholar), Department of CSE, Annamacharya Institute of Technology & Sciences, Rajampet, Kadapa, Andhra Pradesh, India
Dr. A. Subramanyam
Professor, Department of CSE, Annamacharya Institute of Technology & Sciences, Rajampet, Kadapa, Andhra Pradesh, India
Date of Publication : 2017-08-31
License: This work is licensed under a Creative Commons Attribution 4.0 International License.
Page(s) : 695-699
Manuscript Number : CSEIT1172488
Publisher : Technoscience Academy