Real-Time Data Processing with Cloud Technologies
DOI:
https://doi.org/10.32628/CSEIT25112422Keywords:
Cloud Computing, Data Ingestion, Edge Processing, Real-Time Analytics, Serverless ArchitectureAbstract
Real-time data processing, empowered by cloud technologies, has revolutionized how businesses transform raw data streams into actionable insights. This transformative approach enables organizations to respond instantly to changing conditions, identify emerging opportunities and threats, and make decisions based on current information. The evolution from traditional batch processing to modern stream processing architectures represents a technical advancement and a fundamental shift in how enterprises conceptualize their relationship with data. By examining key components, implementation challenges, best practices, industry applications, and future trends, we explore how cloud-based real-time processing creates competitive advantages across diverse sectors while democratizing access to sophisticated data capabilities previously available only to resource-rich organizations.
Downloads
References
Matei Zaharia, et al.,"Apache Spark: A Unified Engine for Big Data Processing," Communications of the ACM, vol. 59, no. 11, pp. 56-65, Oct. 2016. [Online]. Available: https://people.eecs.berkeley.edu/~matei/papers/2016/cacm_apache_spark.pdf
Paris Carbone et al., "Apache Flink™: Stream and Batch Processing in a Single Engine," Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, vol. 38, no. 4, pp. 28-38, Dec. 2015. [Online]. Available: https://asterios.katsifodimos.com/assets/publications/flink-deb.pdf
Jeffrey Dean et al., "MapReduce: Simplified Data Processing on Large Clusters," in Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation, vol. 6, pp. 137-150, 2004. [Online]. Available: https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf
Nathan Marz et al., "Big Data: Principles and best practices of scalable real-time data systems," Manning Publications, 2014. [Online]. Available: https://www.datascienceassn.org/sites/default/files/Big%20Data%20Principles%20and%20Best%20Practices.pdf
Tyler Akidau, "The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing," Proceedings of the VLDB Endowment, vol. 8, no. 12, pp. 1792-1803, Aug. 2015. [Online]. Available: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43864.pdf
Alexander S. Suleykin et al., "Designing Data-Intensive Application System for Production Plans Data Processing and Near Real-Time Analytics," 8th International Conference on Control, Decision and Information Technologies (CoDIT), 2022. [Online]. Available: https://ieeexplore.ieee.org/document/9804133
Jeyhun Karimov et al., "Benchmarking Distributed Stream Data Processing Systems," in IEEE 34th International Conference on Data Engineering (ICDE), 2018, pp. 1507-1518. [Online]. Available: https://asterios.katsifodimos.com/assets/publications/icde18-benchmarks.pdf
Sanjeev Kulkarni et al., "Twitter Heron: Stream Processing at Scale," in Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, 2015, pp. 239-250. [Online]. Available: https://pages.cs.wisc.edu/~shivaram/cs744-readings/Heron.pdf
Michael Stonebraker, et al., "The 8 Requirements of Real-Time Stream Processing " ACM SIGMOD Record, vol. 34, no. 4, pp. 42-47, Dec. 2005. [Online]. Available: https://cs.brown.edu/~ugur/8rulesSigRec.pdf
Eugene Siow et al., "Analytics for the Internet of Things: A Survey," ACM Computing Surveys, Vol. 1, No. 1, Article 1. Publication date: January 2018. [Online]. Available: https://eprints.soton.ac.uk/419347/1/IoTSurveyCSUR.pdf
Weisong Shi et al., "Edge Computing: Vision and Challenges," IEEE Internet Of Things Journal, Vol. 3, No. 5, October 2016. [Online]. Available: https://cse.buffalo.edu/faculty/tkosar/cse710_spring20/shi-iot16.pdf
Eric Jonas et al., "Cloud Programming Simplified: A Berkeley View on Serverless Computing," University of California, Berkeley, Tech. Rep. UCB/EECS-2019-3, Feb. 2019. [Online]. Available: https://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-3.pdf
Downloads
Published
Issue
Section
License
Copyright (c) 2025 International Journal of Scientific Research in Computer Science, Engineering and Information Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.