Enhancing Accessibility with LSTM-Based Sign Language Detection

Authors

  • Azees Abdul  School of Electronics Engineering, VIT-AP University, Amaravati, Andhra Pradesh, India
  • Adithya Valapa  School of Electronics Engineering, VIT-AP University, Amaravati, Andhra Pradesh, India
  • Abdul Kayom Md Khairuzzaman  School of Electronics Engineering, VIT-AP University, Amaravati, Andhra Pradesh, India

DOI:

https://doi.org/10.32628/CSEIT2390517

Keywords:

Long short-term memory, recurrent neural network, sign language identification

Abstract

Sign language serves as a vital means of communication for the deaf and hard of hearing community. However, identifying sign language poses a significant challenge due to its complexity and the lack of a standardized global framework. Recent advances in machine learning, particularly Long Short-Term Memory (LSTM) algorithms, offer promise in the field of sign language gesture recognition. This research introduces an innovative method that leverages LSTM, a type of recurrent neural network designed for processing sequential input. Our goal is to create a highly accurate system capable of anticipating and reproducing sign language motions with precision. LSTM's unique capabilities enhance the recognition of complex gestures by capturing the temporal relationships and fine details inherent in sign language. The results of this study demonstrate that LSTM-based approaches outperform existing state-of-the-art techniques, highlighting the effectiveness of LSTM in sign language recognition and their potential to facilitate communication between the deaf and hearing communities.

References

  1. Rastgoo, R., Kiani, K., & Escalera, S. (2021). Sign language recognition: A deep survey. Expert Systems with Applications, 164, 113794.
  2. Cooper, H., Holt, B., & Bowden, R. (2011). Sign language recognition. Visual Analysis of Humans: Looking at People, 539-562.
  3. Von Agris, U., Zieren, J., Canzler, U., Bauer, B., & Kraiss, K. F. (2008). Recent developments in visual sign language recognition. Universal Access in the Information Society, 6, 323-362.
  4. Nair, A. V., & Bindu, V. (2013). A review on Indian sign language recognition. International journal of computer applications, 73(22).
  5. Yang, S., & Zhu, Q. (2017, July). Continuous Chinese sign language recognition with CNN-LSTM. In Ninth international conference on digital image processing (ICDIP 2017) (Vol. 10420, pp. 83-89). SPIE.
  6. Bantupalli, K., & Xie, Y. (2018, December). American sign language recognition using deep learning and computer vision. In 2018 IEEE International Conference on Big Data (Big Data) (pp. 4896-4899). IEEE.
  7. Elhagry, A., & Elrayes, R. G. (2021). Egyptian sign language recognition using CNN and LSTM. arXiv preprint arXiv:2107.13647.
  8. Sharma, S., Gupta, R., & Kumar, A. (2020). Trbaggboost: An ensemble-based transfer learning method applied to Indian sign language recognition. Journal of Ambient Intelligence and Humanized Computing, 2018
  9. Aparna, C., & Geetha, M. (2020). CNN and stacked LSTM model for Indian sign language recognition. In Machine Learning and Metaheuristics Algorithms, and Applications: First Symposium, SoMMA 2019, Trivandrum, India, December 18–21, 2019, Revised Selected Papers 1 (pp. 126-134). Springer Singapore.
  10. Hillis, M. E., Aubrey, B., Blanchet, J., Shao, Q., Zhou, X., Balkcom, D., & Kraemer, D. J. (2022). Overlapping semantic representations of sign and speech in novice sign language learners. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 44, No. 44).
  11. Kozyra, K., Trzyniec, K., Popardowski, E., & Stachurska, M. (2022). Application for Recognizing Sign Language Gestures Based on an Artificial Neural Network. Sensors, 22(24), 9864.
  12. Singh, J., & Singh, D. (2022, October). A Comprehensive Review on Sign Language Recognition Using Machine Learning. In 2022 10th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions)(ICRITO) (pp. 1-6). IEEE.
  13. Nguen, N. T., Sako, S., & Kwolek, B. (2019). Deep CNN-based recognition of JSL finger spelling. In C. L. M. Q. P. H. C. R. E. Perez Garcia H. Sanchez Gonzalez L. (Ed.). Lecture notes in computer science. Springer International Publishing (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics): Vol. 11734 LNAI.
  14. Chen, F. S., Fu, C. M., & Huang, C. L. (2003). Hand gesture recognition using a real-time tracking method and hidden Markov models. Image and vision computing, 21(8), 745-758.
  15. Zakariah, M., Alotaibi, Y. A., Koundal, D., Guo, Y., & Mamun Elahi, M. (2022). Sign language recognition for Arabic alphabets using transfer learning technique. Computational Intelligence and Neuroscience, 2022.
  16. Adeyanju, I. A., Bello, O. O., & Adegboye, M. A. (2021). Machine learning methods for sign language recognition: A critical review and analysis. Intelligent Systems with Applications, 12, 200056.
  17. Shalev-Arkushin, R., Moryossef, A., & Fried, O. (2022). Ham2Pose: Animating Sign Language Notation into Pose Sequences. arXiv preprint arXiv:2211.13613.
  18. Pansare, J. R., Gawande, S. H., & Ingle, M. (2012). Real-time static hand gesture recognition for American sign language (ASL) in complex background. Journal of Signal and Information Processing, 03(03), 364–367.

IInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology

Downloads

Published

2023-10-30

Issue

Volume 9, Issue 5, September-October-2023

Section

Research Articles

License

Copyright (c) IJSRCSEIT

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Azees Abdul, Adithya Valapa, Abdul Kayom Md Khairuzzaman, " Enhancing Accessibility with LSTM-Based Sign Language Detection" International Journal of Scientific Research in Computer Science, Engineering and Information Technology(IJSRCSEIT), ISSN : 2456-3307, Volume 9, Issue 5, pp.130-139, September-October-2023. Available at doi : https://doi.org/10.32628/CSEIT2390517