Numeric Devnagari Sign Language Translator using Two Hands Single Camera Approach

Authors

  • Jayshree Pansare  Department of Computer Engineering, Modern Education Society's College of Engineering, S.P. Pune University, Pune, Maharashtra, India
  • Maya Ingle  School of Computer Science & Information Technology, Devi Ahilya Vishwavidyalaya, Indore, Madhya Pradesh, India

Keywords:

Devnagari Sign Language (DSL), Fingertip Recognition Technique, Two Hands Single Camera approach, Cluttered Background, Devnagari Numbers

Abstract

Recognition of Numeric Devnagari Sign Language (DSL) using Hand Gesture Recognition System (HGRS) has become an essential tool for hearing and speech impaired to interact with commoner’s via a computer system. Our work is on the development of the proposed system Real-time Numeric Devnagari Sign Language Translator (RTNDSLT). The system architecture of RTNDSLT system is mainly comprised of five phases arranged in layered fashion from top to bottom. Vision-based, real-time and static RTNDSLT system works in cluttered background with mixed lighting conditions. This system focuses on fingertip recognition technique, Peak-and-Valley Detection algorithm, and Peak-Point Detection algorithm along with convex hull. RTNDSLT system achieves a recognition rate of 94.13% using Two Hands Single Camera approach and fingertip recognition technique.

References

  1. M. Gharasuie, et. al., "Real-time Dynamic Hand Gesture Recognition using Hidden Markov Models", in Proceedings of 8th Iranian Conference on Machine Vision and Image Processing (MVIP), 2013,pp. 194-199.
  2. F. Wang, et. al., "Simulating a Smartboard by Real-time Gesture Detection in Lecture Videos", IEEE Transactions on Multimedia, vol. 10, no. 5,pp. 926-935, 2008.
  3. H. Zhu and C. Pun, "Real-time Hand Gesture Recognition from Depth Image Sequences", in IEEE Proceedings of International Conference on Computer Graphics, Imaging and Visualization,2012, pp. 49-52.
  4. T. Starner, et. al., "Real-time American Sign Language Recognition using Desk and Wearable Computer based Video", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1371-1375, 1998.
  5. F. Chen, et. al., "Hand Gesture Recognition using a Real-time Tracking Method and Hidden Markov Models", Image and Vision Computing, vol. 21,pp. 745-758, 2003.
  6. D. Wu, et. al., "Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no.8, pp.1583-1597, 2016.
  7. N. Dardas and N. Georganas, "Real-time Hand Gesture Detection and Recognition using Bag-of-Features and Support Vector Machine Techniques", IEEE Transactions on Instrumentation and Measurement, vol. 60, no. 11, pp. 3592-3607, 2011.
  8. R. Zhang, et. al., "Hand Gesture Recognition with SURF-BOF based on Gray Threshold Segmentation," in IEEE Proceedings of 13th International Conference on Signal Processing (ICIP),2016, pp. 118-122.
  9. H. Gamal, et. al., "Hand Gesture Recognition using Fourier Descriptors", in IEEE Proceedings of 8th International Conference on Computer Engineering and Systems (ICCES), 2013, pp. 274-279.
  10. Z. Liu, et. al., "Research on Chinese-Japanese Sign Language Translation System", in IEEE Proceedings of 5th International Conference on Frontier of Computer Science and Technology, 2010, pp. 640-645.
  11. J. Pansare, et. al., "Real-time Static Hand Gesture Recognition System using HCI For Recognition of Numbers", International Journal of Advanced Research in Computer Science, vol. 4, no. 4, pp. 258-262, 2013.
  12. J. Wachs, et. al., "Cluster Labeling and Parameter Estimation for the Automated Setup of a Hand-Gesture Recognition System", IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 35, no. 6,pp.932-944,2005.
  13. W. Som, et. al., "Novel FPGA Implementation of Hand Sign", IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 1, pp. 153-166, 2015.
  14. P. Dugenie, et. al., "Toward Assessing Subjective Quality of Service of Conversational Mobile Multimedia Applications Delivered over the Internet: A Methodology Study", IEEE Transactions on Multimedia, vol. 4, no. 1, pp.59-67,2002.
  15. S. Nikhil, et. al., "Design and Development of a DSP Processor based Reconfigurable Hand Gesture Recognition System for Real-time Applications", in IEEE Proceedings of International Conference on Signal and Image Processing, 2010,pp.39-44.
  16. G. Mao, et. al., "Real-time Hand Detection and Tracking against Complex Background", in IEEE Proceedings of 5th International Conference on Intelligent Hiding and Multimedia Signal Processing, 2009, pp. 905-908.
  17. Y. Zhu, et. al., "Toward Real-time Human-Computer Interaction with Continuous Dynamic Hand Gestures", in IEEE Proceedings of 4th International Conference on Automatic Face and Gesture Recognition, 2000, pp. 544-549.
  18. Y. Guan and M. Zheng, "Real-time 3D Pointing Gesture Recognition for Natural HCI", in IEEE Proceedings of 7th World Congress on Intelligent Control and Automation, 2008, pp.2433-2436.
  19. J. Wei, et. al., "The Hand Shape Recognition of Human Computer Interaction with Artificial Neural Network", in IEEE Proceedings of International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, 2009, pp. 350-354.
  20. A. Dib, "A Real-time Visual SLAM for RGB-D Cameras based on Chamfer Distance and Occupancy Grid", in IEEE/ACME Proceedings of International Conference on Advanced Intelligent Mechatronics, 2014, pp. 652-657.
  21. Y. Fang, et. al., "A Real-time Hand Gesture Recognition Method", in Proceedings of International Conference on Electronics, Communications, and Control (ICECC), 2011, pp. 2475-2478.
  22. S. Ahmad, "Real-time Rotation Invariant Static Hand Gesture Recognition using an Orientation based Hash Code", in IEEE Proceedings of International Conference on Informatics, Electronics,and Vision (ICIEV),2013,pp.1-6.
  23. A. Chan, et. al., "Real-time Tracking of Hand Gestures for Interactive Game Design", in IEEE Proceedings of International Symposium on Industrial Electronics (ISIE), 2009, pp. 98-103.
  24. J. Lee, et. al., "Real-time Gesture Interface based on Event-Driven Processing from Stereo Silicon Retinas", IEEE Transactions on Neural Networks and Learning Systems, vol.25,no.12,pp.2250-2263,2014.
  25. P. Huang, et. al., "Real-time Stereo Matching For 3D Hand Gesture Recognition", in IEEE Proceedings of International Society Design Conference (ISOC), 2012, pp. 29-32.
  26. Y. Yao and C. Li, "A Framework for Real-time Hand Gesture Recognition in Uncontrolled Environments with Partition Matrix Model based on Hidden Conditional Random Fields",in IEEE Proceedings of International Conference on Systems, Man, and Cybernetics, 2013,pp.1205-1210.
  27. N. Bhat, "Real-time Robust Hand Gesture Recognition and Visual Servoing", in IEEE Proceedings of Annual India Conference (INDICON), 2012, pp. 1153-1157.
  28. J. Lee, et. al., "Real-time Gesture Interface based on Event-Driven Processing from Stereo Silicon Retinas", IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 12,pp. 2250-2263, 2014.
  29. P. Neto, et. al., "Real-time and Continuous Hand Gesture Spotting: an Approach based on Artificial Neural Networks", in IEEE Proceedings of International Conference on Robotics and Automation, 2013, pp. 178-183.
  30. X. Li, et. al., "Hand Gesture Recognition by Stereo Camera using the Thinning Method", in IEEE Proceedings of International Conference on Multimedia Technology, 2011, pp. 3077-3080.
  31. R. Mohana and A. Reddy, "Signer-Independent SLR System using PCA and Multi-class SVM", Praise Worthy Prize International Review on Computers and Software (IRESOE), vol. 9, no. 12, 2014. doi: https://doi.org/10.15866/irescos.v9i12.4838
  32. J. Pansare and M. Ingle, "Real-time Static Hand Gesture Recognition for American Sign Language (ASL) in Complex Background", Journal of Signal and Information Processing, vol. 3, no. 3,pp. 364-367, 2012.
  33. J. R. Pansare and M. Ingle, "A Real-time Devnagari Sign Language Recognizer(α-DSLR) for Devnagari Script", Worlds Conference on Smart Trends in Systems, Security and Sustainability(WS4 2017), 2017.
  34. J. R. Pansare and M. Ingle, "Comprehensive Performance Study of Existing Techniques in Hand Gesture Recognition System for Sign Languages", International Journal of Computer Science and Information Technology, vol. 7, no. 3,pp. 1343-1347, 2016.
  35. J. R. Pansare and M. Ingle, " Vision-based approach for American Sign Language Recognition using Edge Orientation Histogram", IEEE, pp. 86-90, 2016. doi:10.1109/ICIVC.2016.7571278
  36. J. R. Pansare, et al., "Real-time static hand gesture recognition system in complex background that uses number system of Indian Sign Language," Int. J. Adv. Res. Comput. Eng. & Technology, vol. 2, no. 3,pp. 1086-1090, 2013.
  37. J. R. Pansare, et al., "Real-time static Devnagri Sign Language translation using histogram," International Journal of Advanced Research in Computer Engineering & Technology, vol. 2, no. 4,pp. 1-5, 2013.
  38. J. R. Pansare and M. Ingle, "2D Hand Gesture Numeric Devnagari Sign Language Analyzer based on Two Cameras", International Conference on Intelligent Human Computer Interaction 2016 (IHCI 2016), Springer LNCS 10127, pp. 148-160, 2017. doi: 10.1007/978-3-319-52503-7_12.

IInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology

Downloads

Published

2018-01-27

Issue

Volume 3, Issue 1, January-February-2018

Section

Research Articles

License

Copyright (c) IJSRCSEIT

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Jayshree Pansare, Maya Ingle, " Numeric Devnagari Sign Language Translator using Two Hands Single Camera Approach, IInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology(IJSRCSEIT), ISSN : 2456-3307, Volume 3, Issue 1, pp.403-418 , January-February-2018.