Work flow Based Big Data Management in the Cloud Environment

Authors

  • Medarametla Venkata Sairam  PG Scho1lar, Department of CSE, Amrita Sai Institute of Science and Technology, Accredited by NAAC with 'A' Grade, Approved by AICTE, New Delhi, Affiliated to Jntu, Kakinada, Paritala, Krishna District, Andhra Pradesh, India
  • M. Sivanjaneyulu  Associate Prof, Department of CSE, Amrita sai Institute Of Science and Technology Accredited by NAAC with 'A' Grade, Approved by AICTE, New Delhi, India

Keywords:

Big Data, Scientific Workflows, Cloud Computing, Geographically Distributed, Data Management

Abstract

At the point when the workload of amanagement increments quickly, existing methodologies can't respond to the rising execution prerequisite. To proficiently due to either incorrectness of adjustment choices or the moderate procedure of changes, both of which may come about lacking Resource provisioning. The fundamental idea of this paper is capacity to include or expel the cloud Resource provisioning. To enhance the Quality of Service in the Resource management. Resource management arrangements and target independently in every activity. Huge scale issues are dealt with in internet planning the choices in regards to how to plan errands are finished amid the runtime of the framework. The planning choices depend on the projects needs which are either doled out powerfully or statically. Static need driven algorithms apportion preset needs to the projects by the beginning of the framework. Dynamic need driven algorithms dole out the needs to projects amid runtime. An online algorithm is compelled to settle on choices that may later turn out not to be ideal, and the investigation of online algorithms has concentrated on the nature of basic leadership that is conceivable in this setting. Online Resource arrangement creates frameworks to anticipate the dynamic Resource request of Resources and guide the position procedure considers limiting the long-term directing expense between Resources.

References

  1. "Cloud Computing and High-Energy ParticlePhysics: How ATLAS Experiment at CERN UsesGoogle Compute Engine in the Search for New Physics at LHC," https://developers.google.com/events/io/sessions/333315382.
  2. A. Costan, R. Tudoran, G. Antoniu, and G. Brasche,"Tomusblobs: scalable data-intensive processing onazure clouds," Concurrency and Computation: Practiceand Experience, 2013.
  3. T. J. Hacker, B. D. Noble, and B. D. Athey, "Adaptive data blockscheduling for parallel TCP streams," in Proc. 14th IEEE HighPerform. Distrib. Comput., 2005, pp. 265–275.
  4. W. Liu, B. Tieman, R. Kettimuthu, and I. Foster, "A data transferframework for Large-scale science experiments," in Proc. 19thACM Int. Symp. High Perform. Distrib. Comput., 2010, pp. 717–724.
  5. C. Raiciu, C. Pluntke, S. Barre, A. Greenhalgh, D. Wischik, and M.Handley, "Data center networking with multipath tcp," in Proc. 9thACM SIGCOMM Workshop Hot Topics Netw., 2010, pp. 10:1–10:6.
  6. W. Liu, B. Tieman, R. Kettimuthu, and I. Foster, "A data transferframework for Large-scale science experiments," in Proc. 19thACM Int. Symp. High Perform. Distrib. Comput., 2010, pp. 717–724.
  7. P. Carns, W. B. Ligon, R. B. Ross, and R. Thakur, "PVFS: A parallelfile system for linux clusters," in Proc. 4th Annu. Linux ShowcaseConf., 2000, pp. 317–327.
  8. R. L. Grossman, Y. Gu, M. Sabala, and W. Zhang, "Compute andstorage clouds using wide area high performance networks,"Future Gener. Comput. Syst., vol. 25, pp. 179–183, 2009.
  9. R. Tudoran, O. Nano, I. Santos, A. Costan, H. Soncu, L. Bouge, andG. Antoniu, "JetStream: Enabling high performance event streamingacross cloud Data-centers," in Proc. 8th ACM Int. Conf. Distrib. Event-Based Syst., 2014, pp. 23–34.
  10. R. Tudoran, A. Costan, R. R. Rad, G. Brasche, andG. Antoniu, "Adaptive file management for scientificworkflows on the azure cloud," in BigData Conference,2013, pp. 273–281.
  11. R. Tudoran, A. Costan, R. Wang, L. Boug´e, and G.Antoniu, "Bridging data in the clouds: An environment awaresystem for geographically distributed datatransfers," in Proceedings of the 14th IEEE/ACMCCGrid 2014, 2014. Online]. Available: http:// hal.inria.fr/hal-00978153
  12. H.Hiden, S. Woodman, P.Watson, and J.Cała,"Developing cloud applications using the e-sciencecentral platform." in Proceedings of Royal Society A,2012.
  13. K. R. Jackson, L. Ramakrishnan, K. J. Runge, andR. C. Thomas, "Seeking supernovae in the clouds: aperformance study," in Proceedings of the 19th ACMInternational Symposium on High PerformanceDistributed Computing, 2010, pp. 421–429.
  14. A. Greenberg, J. Hamilton, D. A. Maltz, and P.Patel, "The cost of a cloud: research problems in datacenter networks," SIGCOMM Comput. Commun. Rev.,vol. 39, no. 1, pp. 68–73, Dec. 2008.
  15. "Azure Successful Stories," http://www.windowsazure.com/enus/ case studies/archive/.

Downloads

Published

2018-02-28

Issue

Section

Research Articles

How to Cite

[1]
Medarametla Venkata Sairam, M. Sivanjaneyulu, " Work flow Based Big Data Management in the Cloud Environment , IInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology(IJSRCSEIT), ISSN : 2456-3307, Volume 3, Issue 1, pp.845-850, January-February-2018.