Demystifying Multimodal AI: A Technical Deep Dive
DOI:
https://doi.org/10.32628/CSEIT2410612394Keywords:
Multimodal AI, Cross-modal Learning, Neural Embeddings, Visual-Linguistic Understanding, Semantic AlignmentAbstract
This article explores the transformative impact of multimodal AI systems in bridging diverse data types and processing capabilities. It examines how these systems have revolutionized various domains through their ability to handle multiple modalities simultaneously, from visual-linguistic understanding to complex search operations. The article delves into the technical foundations of multimodal embeddings, analyzes leading models like CLIP and MUM, and investigates their real-world applications across different sectors. Through a detailed examination of current implementations, challenges, and future directions, this article provides insights into how multimodal AI reshapes our interaction with digital information while highlighting its potential and limitations in addressing complex real-world scenarios.
Downloads
References
Chaoyou Fu et al., "MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs," arXiv:2411.15296 [cs.CV], 22 Nov 2024. Available: https://arxiv.org/abs/2411.15296
Maciej Żelaszczyk, Jacek Mańdziuk, "Cross-modal text and visual generation: A systematic review. Part 1: Image to text" Information Fusion, Volume 93, May 2023, Pages 302-329. Available: https://www.researchgate.net/publication/220433955_Methods_for_measuring_search_engine_performance_over_time DOI: https://doi.org/10.1016/j.inffus.2023.01.008
Muhammad Arslan Manzoor et al., "Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications," arXiv:2302.00389v2 [cs.AI] 1 Mar 2024. Available: https://arxiv.org/pdf/2302.00389
Zain Hasan, "Multimodal Embedding Models," Weaviate, June 27, 2023. Available: https://weaviate.io/blog/multimodal-models
Alec Radford et al., "Learning Transferable Visual Models From Natural Language Supervision," arXiv:2103.00020 [cs.CV], 26 Feb 2021. Available: https://arxiv.org/abs/2103.00020
GTech, "Google MUM: The Multitask Unified Model Revolutionizing Search Understanding," Oct 21, 2024. Available: https://www.gtechme.com/insights/google-mum-the-multitask-unified-model-revolutionizing-search-understanding/
Parminder Kaur, Husanbir Singh Pannu, Avleen Kaur Malhi, "Comparative analysis on cross-modal information retrieval: A review," Computer Science Review, Volume 39, February 2021, 100336. Available: https://www.sciencedirect.com/science/article/abs/pii/S1574013720304366 DOI: https://doi.org/10.1016/j.cosrev.2020.100336
Euiju Jeong et al., "A Multimodal Recommender System Using Deep Learning Techniques Combining Review Texts and Images," Appl. Sci. 2024, 14(20), 9206, 10 October 2024. Available: https://www.mdpi.com/2076-3417/14/20/9206 DOI: https://doi.org/10.3390/app14209206
Stellarix, "Multimodal AI: Bridging Technologies, Challenges, and Future," Jul 5, 2024. Available: https://stellarix.com/insights/articles/multimodal-ai-bridging-technologies-challenges-and-future/
Katerina Mangaroska et al., "Challenges and opportunities of multimodal data in human learning: The computer science students' perspective," Wiley, 02 March 2021. Available: https://onlinelibrary.wiley.com/doi/10.1111/jcal.12542 DOI: https://doi.org/10.1111/JCAL.12542/v2/response1
Downloads
Published
Issue
Section
License
Copyright (c) 2024 International Journal of Scientific Research in Computer Science, Engineering and Information Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
