Big Data, Technologies and Trends A Study

Authors(2) :-Jayamma Rodda, R. VijayaKumari

Big Data is the most droned terms among Scholars and trade. Sprouting Big Data applications has become gradually more significant in the last few years. In actuality, quiet a few organizations from dissimilar sectors depend all the time more on knowledge extracted from giant volume of data. The different types of users produce massive quantity of data which is not alike in features. The term of big data is referring big data set coming from various sources in the form of structure and non structured data. They need to cater to find out the useful information from the massive, noisy data. However, in Big Data context, traditional data techniques and platforms are less efficient. They show a slow responsiveness and lack of scalability, performance and accuracy. Big Data aims to help to select and adopt the right combination of different Big Data technologies according to their technological needs and specific applications' requirements. The objective of this paper is to provide a simple, comprehensive and brief introduction of Big Data and its technologies. This paper may not cover every dimension of Big Data only few of its essential aspects are covered.

Authors and Affiliations

Jayamma Rodda
Assistant Professor, Department of Master of Computer Science, KBN College, Vijayawada, Andhra Pradesh, India
R. VijayaKumari
Assistant Professor, Department of Computer Science and Applications, Krishna University, Machilipatnam, Andhra Pradesh, India

Big Data, Hadoop, HDFS, Map Reduce, HD Insight, No SQL, Poly base, Presto, PIG, HIVE, and R

  1. Mark Kerzner and SujeeManiyam, "Hadoop Illuminated," https://github.com/hadoop-illuminated/hadoop-book, 2013, Accessed on Sept. 20, 2015.
  2. L Douglas, "3d data management: Controlling data volume, velocity and variety," Gartner, Retrieved 6 (2001).
  3. IBM What is big data? - Bringing big data to the enterprise. http://www-01.ibm.com/software/in/data/bigdata/, Accessed on Sept. 20, 2015.
  4. M A. Beyer and L. Douglas, "The importance of big data: A definition," Stamford, CT: Gartner, 2012.
  5. IBM Big Data &Analytics Hub, http://www.ibmbigdatahub.com/infographic/four-vs-big-data, Accessed on Sept. 20, 2015.
  6. J S. Ward and A. Barker, "Undefined By Data: A Survey of BigData Definitions," http://arxiv.org/abs/1309.5821v1.
  7. Stimmel, C.L., 2014. Big Data Analytics Strategies for the Smart Grid. CRC Press.Stoianov, N., Uruea, M., Niemiec, M., Machnik, P., Maestro, G., 2013. Integrated security infrastructures for law enforcement agencies. Multimedia Tools App.,1-16
  8. Nambiar, R., Bhardwaj, R., Sethi, A., Vargheese, R., 2013. A look at challenges and opportunities of Big Data analytics in healthcare. In: In: 2013 IEEE International Conference on Big Data. IEEE, pp.17-22.
  9. Chen, M., Mao, S., Zhang, Y., Leung, V.C., 2014b. Big Data: Related Technologies, Challenges and Future Prospects. Springer.
  10. Rajaraman, V., 2016. Big data analytics. Resonance 21, 695-716.
  11. Tom White, "Hadoop: The definitive guide," O'Reilly Media, Inc.,2012.
  12. S. Ghemawat, H. Gobioff and ST Leung, "The Google file system," in ACM SIGOPS operating systems review, vol. 37, no. 5, ACM,2003.
  13. J. Dean and S. Ghemawat, "MapReduce: simplified data processing on large clusters," in Proc. 6th Symposium on Opearting Systems Design & Implementation, 2004.
  14. ApacheHadoop, http://hadoop.apache.org
  15. Sort Benchmark, http://sortbenchmark.org/
  16. Yahoo! Hadoop Tutorial, http://developer.yahoo.com/hadoop/tutorial/index.html, Accessed onSept. 20, 2015.
  17. HDFS Architecture Guide, https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html, Accessedon Sept. 20, 2015.
  18. Sudhakar Singh, RakhiGarg and P K Mishra, "Review of Apriori Based Algorithms on MapReduce Framework," in Proc. International Conference on Communication and Computing (ICC - 2014), Elsevier Science and Technology Publications, 2014.
  19. MapReduce Tutorial, http://hadoop.apache.org/docs/current/hadoop- mapreduce-client/hadoop-mapreduce-client- core/MapReduceTutorial.html, Accessed on Sept. 20, 2015.
  20. K-H. Lee, Y-J. Lee, H. Choi, Y. D. Chung and B. Moon, "Parallel Data Processing with MapReduce: A Survey," in ACM SIGMOD Record, vol. 40, no. 4, pp. 11-20, (2011).
  21. Hadoop Tutorials, http://hadooptutorials.co.in/tutorials/hadoop/understanding-hadoop- ecosystem.html, Accessed on Sept. 20, 2015.
  22. Hadoop Architecture Overview, http://ercoppa.github.io/HadoopInternals/HadoopArchitectureOverview.html, Accessed on Sept. 20, 2015.
  23. IBM developer Works, http://www.ibm.com/developerworks/library/l-hadoop-1/,Accessed on Sept. 20, 2015.
  24. R. P. Padhy, "Big Data Processing with Hadoop-MapReduce in Cloud Systems," in International Journal of Cloud Computing and Services Science (IJ-CLOSER), vol. 2, no. 1, pp.16-27, 2013.+.
  25. https://customers.microsoft.com/en-us/story/pros
  26. https://azure.microsoft.com/en-us/blog/announcing-public-preview-of-apache-kafka-on-hdinsight-with-azure-managed-disks/
  27. https://www.mongodb.com/nosql-explained
  28. https://en.wikipedia.org/wiki/Presto_(SQL_query_engine)
  29. Joab Jackson (November 6, 2013). "Facebook goes open source with query engine for big data". Computer World.Retrieved April 26, 2017.
  30. Jordan Novet (June 6, 2013). "Facebook unveils Presto engine for querying 250 PB data warehouse". Giga Om.Retrieved April 26, 2017.
  31. Eva Tse, ZhenxiaoLuo, NezihYigitbasi (October 7, 2014). "Using Presto in our Big Data Platform on AWS".Netflix technical blog.Retrieved April 26, 2017.
  32. Doug Henschen (March 5, 2015). "Airbnb Boosts Presto SQL Query Engine ForHadoop". Information Week.Retrieved April 26, 2017.
  33. James Mayfield (March 4, 2015). "Airpal: a Web UI for PrestoDB". Airbnb blog post.Archived from the original on March 6, 2015.Retrieved April 26, 2017.
  34. https://en.wikipedia.org/wiki/Presto_(SQL_query_engine)
  35. https://en.wikipedia.org/wiki/NoSQL
  36. https://docs.microsoft.com/en-us/sql/relational-databases/polybase/polybase-guide?view=sql-server-2017
  37. https://www.microsoft.com/en-us/microsoft-365/blog/2016/06/23/excel-and-big-data/
  38. Vohra, D., 2016. Using apache sqoop. In: Pro Docker. Springer, pp.151-183.
  39. Jain, A., 2013. Instant Apache Sqoop.Packt Publishing Ltd..
  40. Ahmed Oussous , Fatima-Zahra Benjelloun , Ayoub Ait Lahcen , Samir Belfkih "Big Data technologies: A survey"LGS, National School of Applied Sciences (ENSA), Ibn Tofail University, Kenitra, Morocco LRIT, Unit associe au CNRST URAC 29, Mohammed V University in Rabat, Morocco Journal of King Saud University - Computer and Information Sciences
  41. Sudhakar Singh a,*, Pankaj Singh b, Rakhi Garg c, P K Mishra a "Big Data: Technologies, Trends and Applications" / (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 6 (5) , 2015, 4633-4639
  42. Rachit Singhal, Mehak Jain and Shilpa Gupta " Comparative Analysis of Big Data Technologies" International Journal of Applied Engineering Research ISSN 0973-4562 Volume 13, Number 6 (2018) pp. 3822-3830 Research India Publications. http://www.ripublication.com 3822

Publication Details

Published in : Volume 3 | Issue 7 | September-October 2018
Date of Publication : 2018-10-30
License:  This work is licensed under a Creative Commons Attribution 4.0 International License.
Page(s) : 400-414
Manuscript Number : CSEIT183793
Publisher : Technoscience Academy

ISSN : 2456-3307

Cite This Article :

Jayamma Rodda, R. VijayaKumari, "Big Data, Technologies and Trends A Study", International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN : 2456-3307, Volume 3, Issue 7, pp.400-414, September-October-2018. |          | BibTeX | RIS | CSV

Article Preview