American Sign Language Recognition and Generation : A CNN-based Approach

Authors

  • Pusti Sheth  Department of Information Technology, Dwarkadas J. Sanghvi College of Engineering, Mumbai, Maharashtra, India
  • Ronik Dedhia  Department of Information Technology, Dwarkadas J. Sanghvi College of Engineering, Mumbai, Maharashtra, India
  • Akshit Chheda  Department of Information Technology, Dwarkadas J. Sanghvi College of Engineering, Mumbai, Maharashtra, India
  • Dr. Vinaya Sawant  Head of Department, Department of Information Technology, Dwarkadas J. Sanghvi College of Engineering, Mumbai, Maharashtra, India

DOI:

https://doi.org/10.32628/CSEIT23902103

Keywords:

American Sign Language (ASL), Convolutional Neural Networks (CNNs), Deep learning, Accuracy rates

Abstract

Although not a global language, sign language is an essential tool for the deaf community. Communication between these communities and hearing population is severely hampered by this, as human-based interpretation can be both costly and time-consuming. In this paper, we present a real-time American Sign Language (ASL) generation and recognition system that makes use of Convolutional Neural Networks and deep learning (CNNs). Despite differences in lighting, skin tones, and backdrops, our technology is capable of correctly identifying and generating ASL signs. We trained our model on a large dataset of ASL signs in order to obtain a high level of accuracy. Our findings show that, with accuracy rates of 98.53% and 98.84%, respectively, our system achieves high accuracy rates in both training and validation. Our approach uses the advantages of CNNs to accomplish quick and precise recognition of individual letters and words, making it particularly effective for sign fingerspelling recognition. We believe that our technology has the ability to transform communication between the hearing community and the deaf and hard-of-hearing communities by providing a dependable and cost-effective way of sign language interpretation. Our method could help people who use sign language communicate more easily and live better in a range of environments, including schools, hospitals, and public places.

References

  1. Starner, T., Pentland, A.: Real-time american sign language recognition from video using hidden Markov models. Proceedings of International Symposium on Computer Vision - ISCV, 265–270 (1995)
  2. Jebali, M., Dalle, P., Jemni, M.: Extension of hidden markov model for recognizing large vocabulary of sign language. International Journal of Artificial Intelligence Applications 4 (2013) https://doi.org/10.5121/ijaia.2013.4203
  3. Suk, H.-I., Sin, B.-K., Lee, S.-W.: Hand gesture recognition based on dynamic bayesian network framework. Pattern Recognition 43, 3059–3072 (2010) https: doi.org/10.1016/j.patcog.2010.03.016
  4. Mekala, P., Gao, Y., Fan, J., Davari, A.: Real-time sign language recognition based on neural network architecture. 2011 IEEE 43rd Southeastern Symposium on System Theory, 195–199 (2011)
  5. Admasu, Y.F., Raimond, K.: Ethiopian sign language recognition using artificial neural network. 2010 10th International Conference on Intelligent Systems Design and Applications, 995–1000 (2010)
  6. Atwood, J., Farrell, J.: American sign language recognition system. (2012)
  7. Pigou, L., Dieleman, S., Kindermans, P.-J., Schrauwen, B.: Sign language recognition using convolutional neural networks, vol. 8925, pp. 572–578 (2015). https://doi.org/10.1007/978-3-319-16178-5 40
  8. He, S.: Research of a sign language translation system based on deep learning. 2019 International Conference on Artificial Intelligence and Advanced Manufacturing (AIAM), 392–396 (2019)
  9. Geetha, M.K., ManjushaU, C.: A vision based recognition of indian sign language alphabets and numerals using b-spline approximation. (2012)
  10. Herath, H.C.M., W.A.L.V.Kumari, Senevirathne, W.A.P.B., Dissanayake, M.: Image based sign language recognition system for sinhala sign language. (2013)
  11. Huang, J., Zhou, W.-g., Li, H., Li, W.: Sign language recognition using 3d convolutional neural networks. 2015 IEEE International Conference on Multimedia and Expo (ICME), 1–6 (2015)

Downloads

Published

2023-08-30

Issue

Section

Research Articles

How to Cite

[1]
Pusti Sheth, Ronik Dedhia, Akshit Chheda, Dr. Vinaya Sawant, " American Sign Language Recognition and Generation : A CNN-based Approach " International Journal of Scientific Research in Computer Science, Engineering and Information Technology(IJSRCSEIT), ISSN : 2456-3307, Volume 9, Issue 4, pp.241-250, July-August-2023. Available at doi : https://doi.org/10.32628/CSEIT23902103