Transfer Learning-Based Recognition of Bhutanese Sign Language Digits Using Deep CNNs
DOI:
https://doi.org/10.32628/CSEIT2612133Keywords:
Bhutanese sign digit, convolutional neural network, MobileNet, transfer learning, ResNet50, VGG16Abstract
The hearing and speech-impaired community uses hand gesture-based communication media to communicate with general public. However, the general public finds it difficult to communicate with them due to their difficulties in understanding sign digits, thereby creating a communication gap between the general public and the hearing-impaired community. Therefore, this paper proposes three pre-trained (VGG16, ResNet50, MobileNet) models to train models on the Bhutanese sign digit dataset. In this study, two different datasets (Bhutanese sign digit and Turkish sign digit) were merged and used to train the models. The rationale for merging the datasets is that both datasets use the same representation of sign gestures and have different variations of images. The dataset was split into train and test sets with a ratio of 80:20. The VGG16 network architecture outperformed the other two models with the training and testing accuracy of 96.72% and 95.85%. The trained model was integrated with the Django framework to create a web application for digit recognition.
Downloads
References
L. Wangmo, ‘Physical impairment cases see steady rise – Business Bhutan’. Accessed: Oct. 12, 2021. [Online]. Available: https://businessbhutan.bt/physical-impairment-cases-see-steady-rise/
A. L. C. Barczak, N. H. Reyes, M. Abastillas, A. Piccio, and T. Susnjak, ‘A New 2D Static Hand Gesture Colour Image Dataset for ASL Gestures’, p. 9.
P. Kiranalli and S. R., ‘Indian Sign Language Numeral Recognition - An Image Processing Approach’, IJCA, vol. 146, no. 7, pp. 24–27, Jul. 2016, doi: 10.5120/ijca2016910866. DOI: https://doi.org/10.5120/ijca2016910866
O. Sevli̇ and N. Kemaloğlu, ‘Turkish sign language digits classification with CNN using different optimizers’, International Advanced Researches and Engineering Journal, vol. 4, no. 3, pp. 200–207, Dec. 2020, doi: 10.35860/iarej.700564. DOI: https://doi.org/10.35860/iarej.700564
Md. M. Hasan and Sk. M. M. Ahsan, ‘Bangla Sign Digits Recognition Using HOG Feature Based Multi-Class Support Vector Machine’, in 2019 4th International Conference on Electrical Information and Communication Technology (EICT), Dec. 2019, pp. 1–5. doi: 10.1109/EICT48899.2019.9068832. DOI: https://doi.org/10.1109/EICT48899.2019.9068832
D. Tasmere, B. Ahmed, and M. M. Hasan, ‘Bangla Sign Digits: A Dataset For Real Time Hand Gesture Recognition’, in 2020 11th International Conference on Electrical and Computer Engineering (ICECE), Dec. 2020, pp. 186–189. doi: 10.1109/ICECE51571.2020.9393070. DOI: https://doi.org/10.1109/ICECE51571.2020.9393070
K. Wangchuk, P. Riyamongkol, and R. Waranusast, ‘Real-time Bhutanese Sign Language digits recognition system using Convolutional Neural Network’, ICT Express, vol. 7, no. 2, pp. 215–220, Jun. 2021, doi: 10.1016/j.icte.2020.08.002. DOI: https://doi.org/10.1016/j.icte.2020.08.002
Md. S. Alom, Md. J. Hasan, and Md. F. Wahid, ‘Digit Recognition in Sign Language Based on Convolutional Neural Network and Support Vector Machine’, in 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), Dec. 2019, pp. 1–5. doi: 10.1109/STI47673.2019.9067999. DOI: https://doi.org/10.1109/STI47673.2019.9067999
Md. A. Kalam, Md. N. I. Mondal, and B. Ahmed, ‘Rotation Independent Digit Recognition in Sign Language’, in 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE), Feb. 2019, pp. 1–5. doi: 10.1109/ECACE.2019.8679172. DOI: https://doi.org/10.1109/ECACE.2019.8679172
Chollet, Francois, and others, Keras. (2015). GitHub. [Online]. Available: https://github.com/fchollet/keras
G. Bradski, ‘The OpenCV Library’, Dr. Dobb’s Journal of Software Tools, 2000.
K. Simonyan and A. Zisserman, ‘Very Deep Convolutional Networks for Large-Scale Image Recognition’, Apr. 10, 2015, arXiv: arXiv:1409.1556. doi: 10.48550/arXiv.1409.1556.
K. He, X. Zhang, S. Ren, and J. Sun, ‘Deep Residual Learning for Image Recognition’, Dec. 10, 2015, arXiv: arXiv:1512.03385. doi: 10.48550/arXiv.1512.03385.
A. G. Howard et al., ‘MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications’, Apr. 16, 2017, arXiv: arXiv:1704.04861. doi: 10.48550/arXiv.1704.04861.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Scientific Research in Computer Science, Engineering and Information Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
