An Efficient Resource Aware Scheduling Algorithm for Mapreduce Clusters

Authors(4) :-Sharmilarani D, Vinothini K, Ramya V, Shobika R

MapReduce has become a popular model for data-intensive computation in recent years. The schedulers are critical in enhancing the performance of MapReduce/Hadoop in presence of multiple jobs with different characteristics and performance goals. The propose improve the resource aware scheduling technique for Hadoop map-reduce multiple jobs running that aims to improving resource utilization across multiple virtual machines while observing completion time goals. The propose algorithm influences job profiling information to dynamically adjust the number of slots allocation based on job profile and resource utilization on each machine, as well as workload placement across them, to maximize the resource utilization of the cluster. This single node experimental result show the resource aware scheduling that improves job running time and reduce the resource utilization without introducing stragglers.

Authors and Affiliations

Sharmilarani D
Department of Computer Science Engineering, Sri Krishna Institute of Technology, Coimbatore, TamilNadu, India
Vinothini K
Department of Computer Science Engineering, Sri Krishna Institute of Technology, Coimbatore, TamilNadu, India
Ramya V
Department of Computer Science Engineering, Sri Krishna Institute of Technology, Coimbatore, TamilNadu, India
Shobika R
Department of Computer Science Engineering, Sri Krishna Institute of Technology, Coimbatore, TamilNadu, India

Hadoop, Map-Reduce, Resource Aware Scheduling Profiling

  1. G. Z. Guo, G. Fox, and M. Zhou, "Investigation of data locality in mapreduce," in Proc. 12th IEEE/ACM Int. Symp. Cluster, Cloud Grid Comput., May 2012, pp. 419–426.
  2. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica, "Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling," in Proc. 5th Eur. Conf. Comput. Syst., Apr. 2010, pp. 265–278.
  3. J. Jin, J. Luo, A. Song, F. Dong, and R. Xiong, "BAR: An efficient data locality driven task scheduling algorithm for cloud computing," in Proc. 11th IEEE/ACM Int. Symp. Cluster, Cloud Grid Comput., May 2011, pp. 295–304.
  4. M. Ehsan, and R. Sion, "LiPS: A cost-efficient data and task coscheduler for MapReduce," in Proc. IEEE 27th Int. Symp. Parallel Distrib. Process. Workshops PhD Forum, May 2013, pp. 2230–2233.
  5. J. Park, D. Lee, B. Kim, J. Huh, and S. Maeng, "Locality-aware dynamic VM reconfiguration on MapReduce clouds," in Proc. 21st Int. Symp. High-Perform. Parallel Distrib. Comput., Jun. 2012, pp. 27–36.
  6. X. Bu, J. Rao, and C.-Z. Xu, "Interference and locality-aware task scheduling for Mapreduce applications in virtual clusters," in Proc. 22nd Int. Symp. High-Perform. Parallel Distrib. Comput., Jun. 2013, pp. 227– 238.
  7. C. Tian, H. Zhou, Y. He, and L. Zha, "A dynamic mapreduce scheduler for heterogeneous workloads," in Proc. IEEE 8th Int. Conf. Grid Cooperative Comput., 2009, pp. 218–224.
  8. J. Polo, D. Carrera, Y. Becerra, J. Torres, E. Ayguade, M. Steinder, and I. Whalley, "Performance-driven task co-scheduling for mapreduce environments," in Proc. IEEE Netw. Oper. Manage. Symp., 2010, pp. 373–380.

Publication Details

Published in : Volume 2 | Issue 2 | March-April 2017
Date of Publication : 2017-04-30
License:  This work is licensed under a Creative Commons Attribution 4.0 International License.
Page(s) : 517-523
Manuscript Number : CSEIT1722182
Publisher : Technoscience Academy

ISSN : 2456-3307

Cite This Article :

Sharmilarani D, Vinothini K, Ramya V, Shobika R, "An Efficient Resource Aware Scheduling Algorithm for Mapreduce Clusters", International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN : 2456-3307, Volume 2, Issue 2, pp.517-523, March-April-2017.
Journal URL : http://ijsrcseit.com/CSEIT1722182

Article Preview

Follow Us

Contact Us