An Efficient Resource Aware Scheduling Algorithm for Mapreduce Clusters

Authors

  • Sharmilarani D  Department of Computer Science Engineering, Sri Krishna Institute of Technology, Coimbatore, TamilNadu, India
  • Vinothini K  Department of Computer Science Engineering, Sri Krishna Institute of Technology, Coimbatore, TamilNadu, India
  • Ramya V  Department of Computer Science Engineering, Sri Krishna Institute of Technology, Coimbatore, TamilNadu, India
  • Shobika R  Department of Computer Science Engineering, Sri Krishna Institute of Technology, Coimbatore, TamilNadu, India

Keywords:

Hadoop, Map-Reduce, Resource Aware Scheduling Profiling

Abstract

MapReduce has become a popular model for data-intensive computation in recent years. The schedulers are critical in enhancing the performance of MapReduce/Hadoop in presence of multiple jobs with different characteristics and performance goals. The propose improve the resource aware scheduling technique for Hadoop map-reduce multiple jobs running that aims to improving resource utilization across multiple virtual machines while observing completion time goals. The propose algorithm influences job profiling information to dynamically adjust the number of slots allocation based on job profile and resource utilization on each machine, as well as workload placement across them, to maximize the resource utilization of the cluster. This single node experimental result show the resource aware scheduling that improves job running time and reduce the resource utilization without introducing stragglers.

References

  1. G. Z. Guo, G. Fox, and M. Zhou, "Investigation of data locality in mapreduce," in Proc. 12th IEEE/ACM Int. Symp. Cluster, Cloud Grid Comput., May 2012, pp. 419–426.
  2. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica, "Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling," in Proc. 5th Eur. Conf. Comput. Syst., Apr. 2010, pp. 265–278.
  3. J. Jin, J. Luo, A. Song, F. Dong, and R. Xiong, "BAR: An efficient data locality driven task scheduling algorithm for cloud computing," in Proc. 11th IEEE/ACM Int. Symp. Cluster, Cloud Grid Comput., May 2011, pp. 295–304.
  4. M. Ehsan, and R. Sion, "LiPS: A cost-efficient data and task coscheduler for MapReduce," in Proc. IEEE 27th Int. Symp. Parallel Distrib. Process. Workshops PhD Forum, May 2013, pp. 2230–2233.
  5. J. Park, D. Lee, B. Kim, J. Huh, and S. Maeng, "Locality-aware dynamic VM reconfiguration on MapReduce clouds," in Proc. 21st Int. Symp. High-Perform. Parallel Distrib. Comput., Jun. 2012, pp. 27–36.
  6. X. Bu, J. Rao, and C.-Z. Xu, "Interference and locality-aware task scheduling for Mapreduce applications in virtual clusters," in Proc. 22nd Int. Symp. High-Perform. Parallel Distrib. Comput., Jun. 2013, pp. 227– 238.
  7. C. Tian, H. Zhou, Y. He, and L. Zha, "A dynamic mapreduce scheduler for heterogeneous workloads," in Proc. IEEE 8th Int. Conf. Grid Cooperative Comput., 2009, pp. 218–224.
  8. J. Polo, D. Carrera, Y. Becerra, J. Torres, E. Ayguade, M. Steinder, and I. Whalley, "Performance-driven task co-scheduling for mapreduce environments," in Proc. IEEE Netw. Oper. Manage. Symp., 2010, pp. 373–380.

IInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology

Downloads

Published

2017-04-30

Issue

Volume 2, Issue 2, March-April-2017

Section

Research Articles

License

Copyright (c) IJSRCSEIT

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Sharmilarani D, Vinothini K, Ramya V, Shobika R, " An Efficient Resource Aware Scheduling Algorithm for Mapreduce Clusters, IInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology(IJSRCSEIT), ISSN : 2456-3307, Volume 2, Issue 2, pp.517-523, March-April-2017.