Video Summarization using Deep Learning
Keywords:
Object of Interest (OoI); Video Summarization (VSUMM) dataset; Title-based Video Summarization (TVSum) dataset; Own dataset.Abstract
The advancements in digital video technology have empowered video surveillance to play a vital role in ensuring security and safety. Public and private enterprises use surveillance systems to monitor and analyze daily activities. Consequently, a massive volume of data is generated in videos that require further processing to achieve security protocol. Analyzing video content is tedious and a time-consuming task. Moreover, it also requires high- speed computing hardware. The video summarization concept has emerged to overcome these limitations. This paper presents a customized video summarization framework based on deep learning. The proposed framework enables a user to summarize the videos according to the Object of Interest (OoI), for example, person, airplane, mobile phone, bike, and car. Various experiments are conducted to evaluate the performance of the proposed framework on the video summarization (VSUMM) dataset, title-based video summarization (TVSum) dataset, and own dataset. The accuracy of VSUMM, TVSum, and own dataset is 99.6%, 99.9%, and 99.2%, respectively. A desktop application is also developed to help the user summarize the video based on the OoI. in the problem of video summarization, the goal is to select a subset of the input frames conveying the most important information of the input video. The collection of data proves to be a challenging task. The goal of video summarization is to shorten an input video to a summary video which conveys the most important information of the original video.
References
- Security Info Watch, “Data generated by new surveillance cameras to increase exponentially in the coming years,” 2016.
- B. Mahasseni, M. Lam, and S. Todorovic, “Unsupervised Video Summarization with Adversarial LSTM Networks,” in 2017 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2982–2991.
- E. Cosgrove, “One billion surveillance cameras will be watching around the world in 2021, a new study says,” 2019.
- X. Yan, S. Z. Gilani, M. Feng, L. Zhang, H. Qin, and A. Mian, “Self-supervised learning to detect key frames in videos,” Sensors, vol. 20, no. 23, p. 6941, 2020.
- B. Korbar, D. Tran, and L. Torresani, “Scsampler: sampling salient clips from video for efficient action recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6232–6242, IEEE, Seoul, Korea (South), November 2020.
- J. Huo and T. L. van Zyl, “Unique faces recognition in videos,” in Proceedings of the 2020 IEEE 23rd International Conference on Information Fusion (FUSION), pp. 1–7, IEEE, Rustenburg, South Africa, July 2020.
- S. Manna, S. Ghildiyal, and K. Bhimani, “Face recognition from video using deep learning,” in Proceedings of the 2020 5th International Conference on Communication and Electronics Systems (ICCES), pp. 1101–1106, IEEE, Coimbatore, India, June 2020.
- J. Zhang, S. Chen, S. Tian, W. Gong, G. Cai, and Y. Wang, “A crowd counting framework combining with crowd location,” Journal of Advanced Transportation, vol. 2021, Article ID 6664281, 12 pages, 2021.
- W. Ullah, A. Ullah, T. Hussain et al., “Artificial Intelligence of Things-assisted two- stream neural network for anomaly detection in surveillance Big Video Data,” Future Generation Computer Systems, vol. 129, pp. 286–297, 2022.
- S.-H. Zhong, J. Lin, J. Lu, A. Fares, and T. Ren, “Deep semantic and attentive network for unsupervised video summarization,” ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 18, no. 2, pp. 1–21, 2022.
Downloads
Published
Issue
Section
License
Copyright (c) IJSRCSEIT
This work is licensed under a Creative Commons Attribution 4.0 International License.