Leveraging Artificial Intelligence to Optimize ETL Pipelines: Enhancing Efficiency, Accuracy, and Scalability
DOI:
https://doi.org/10.32628/CSEIT251112385Keywords:
Artificial Intelligence in ETL, Self-healing Systems, Predictive Data Maintenance, Intelligent Data Discovery, Cloud Resource OptimizationAbstract
This article explores the transformative impact of Artificial Intelligence on Extract, Transform, Load (ETL) processes in modern enterprise data management. The article examines how AI technologies enhance data extraction through intelligent discovery systems, unstructured data processing, and adaptive web scraping capabilities. It investigates the role of AI in data transformation, including automated cleansing, smart mapping, and real-time quality monitoring. The article further analyzes AI's contribution to optimizing data loading through intelligent scheduling and adaptive performance tuning, examining self-healing capabilities and predictive maintenance in data systems. Additionally, the article evaluates how AI enables scalable and efficient infrastructure management, providing insights into resource optimization and performance enhancement strategies for enterprise data operations.
Downloads
References
BrandBoosters Research, "Global Enterprise Architecture Market Report: Key Trends, Size, and Growth Opportunities," LinkedIn, 2024. [Online]. Available: https://www.linkedin.com/pulse/global-enterprise-architecture-market-report-key-ueycf/
Charles Paul et al., "Leveraging AI and Machine Learning for ETL Process Optimization," ResearchGate, February 2022. [Online]. Available: https://www.researchgate.net/publication/387534347_Leveraging_AI_and_Machine_Learning_for_ETL_Process_Optimization
Artificio, "AI-Driven Data Extraction: Revolutionizing Medical Research," May 13th, 2024. [Online]. Available: https://artificio.ai/blog/ai-driven-data-extraction-revolutionizing-medical-research
Julie Simpson, "Transforming Unstructured Data with AI: The Future of Business Efficiency," Ronin. [Online]. Available: https://www.ronin.consulting/artificial-intelligence/unstructured-data-with-ai/
Oksana Zdrok, "The Critical Role of Data Quality in AI Implementations," Shelf.io, May 2, 2024. [Online]. Available: https://shelf.io/blog/data-quality-in-ai-implementations/
Morgan Sullivan, "Automating the Future: The Rise of Data Mapping Automation Tools," Transcend, February 22, 2024. [Online]. Available: https://transcend.io/blog/automated-data-mapping
Olukunle Oladipupo Amoo et al., "AI-driven warehouse automation: A comprehensive review of systems," ResearchGate, February 2024. [Online]. Available: https://www.researchgate.net/publication/378307805_AI-driven_warehouse_automation_A_comprehensive_review_of_systems
Wei Li and Maolin Tang, "The Performance Optimization of Big Data Processing by Adaptive MapReduce Workflow," IEEE Access, 1 August 2022. [Online]. Available: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9839461
Rajeev Kumar Arora et al., "AI-Driven Self-Healing Cloud Systems: Enhancing Reliability and Reducing Downtime through Event-Driven Automation," Applied Intelligence and Computing, 293–301, 2025. [Online]. Available: https://www.publications.scrs.in/uploads/final_menuscript/eb15d723c07f624f1e2892976bec34c4.pdf
Serhii Leleko and Roman Chupryna, "Predictive Maintenance with Machine Learning: A Complete Guide," SPD Technology, 19.04.2024. [Online]. Available: https://spd.tech/machine-learning/predictive-maintenance/
Ethan Lee, "AI-Driven Cloud Resource Optimization: A Developer's Guide," Dev.io, Oct 7, 2024. [Online]. Available: https://dev.to/vcian/ai-driven-cloud-resource-optimization-a-developers-guide-2h4
Sparity, "The role of Artificial Intelligence (AI) and Machine Learning (ML) in cloud computing." [Online]. Available: https://www.sparity.com/blogs/the-role-of-artificial-intelligence-ai-and-machine-learning-ml-in-cloud-computing/
Downloads
Published
Issue
Section
License
Copyright (c) 2025 International Journal of Scientific Research in Computer Science, Engineering and Information Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.