The Comprehensive Study of Facial Expression Recognition on Video

Authors

  • Ratnalata Gupta  Department of Computer Science, Oriental University, Indore, Madhya Pradesh, India
  • L. K. Vishwamitra  Department of Computer Science, Oriental University, Indore, Madhya Pradesh, India

Keywords:

Facial Expression Recognition, Deep Learning, Classification.

Abstract

In human life, facial expressions and emotions reveal external and internal responses. In human-computer interaction, the video clip plays an important part for extract the emotion of the end-user. In this type of system, it is necessary to observe rapid dynamic changes in the motion of the human face to give the essential response. A real-time application is based on identifying the face and recognizes his expressions such as happy, sad, disgruntled and tiredness, etc. for example Driver exhaustion detection to prevent road accidents on the road. We focus on the comprehensive study of different traditional and recent methodsfor preprocessing, features take out and emotion recognition is employed in facial expression and recognition systems (FER). This paper also describes various Terminologies are used in the Facial expression and recognition system (FER) system. The result compares with the Number of Expressions, algorithm accuracy, and implementation tools. Video-based facial emotion recognition is a very interesting and challenging problem hence the present study provides model complexity, implementation trends, and opportunities for researchers can consider as future research works.

References

  1. Chris Frith,” Role of facial expressions in social interactions”, Phil. Trans. R. Soc, pp 3453–3458, (2009).
  2. S Zhang, X Pan, Y Cui, X Zhao, L Liu., “Learning Affective Video Features for Facial Expression Recognition via Hybrid Deep Learning”, IEEE Access vol 7, pp.32297-32304, (2019).
  3. F. Khan, “Facial Expression Recognition using FacialLandmark Detection and Feature Extraction via Neural Networks”, Xiv:1812.04510v3 [cs.CV], pp.1-7,(2020).
  4. M. Yeasin, B. Bullot, R. Sharma, “Recognition of Facial Expressions and Measurement of Levels of Interest from Video”, IEEE transactions on multimedia, Vol. 8, pp.500- 508(2006).
  5. S Bashyal, GK Venayagamoorthy, “Recognition of facial expressions using Gabor wavelets and learning vector quantization”, Engineering Applications of Artificial Intelligence Elsevier, pp 1056–1064, (2007).
  6. S. Biswas, J. Sil, “An efficient face recognition method using contourlet and curvelet transform” Journal of King Saud University Elsevier, pp. 1-12, (2017).
  7. Ebenezer Owusu, Yongzhao Zhan, Qi Rong Mao, “A neural- AdaBoost based facial expression recognition system” Expert Systems with Applications Elsevier, pp. 3383-3390(2014).
  8. P Viola, M Jones, Rapid Object Detection using a Boosted Cascade of Simple Features”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, pp. 511-518, (2001).
  9. J. Yang, S.Liao, Stan Z. Li “Automatic Partial Face Alignment in NIR Video Sequences”, International Conference on Biometrics Springer, pp. 249–258, (2009).
  10. Hernandez- Matamoros, A. Bonarini, E. Hernandez, M Nakano- Miyatake, H.Meana “ Facial Expression Recognition with Automatic Segmentation of Face Regions using a Fuzzy based Classification Approach” Knowledge-Based Systems, Vol- 110, Pages 1-14,(2016)
  11. Ke Shan, Junqi Guo, Wenwan You, Di Lu, RongfangBie “Automatic Facial Expression Recognition Based on a Deep Convolutional-Neural-Network Structure”¸ IEEE 15thInternational Conference on Software Engineering Research, Management and Applications (SERA) ¸ pp: 123-128(2017).
  12. Caifeng Shan, Shaogang Gong, Peter W. McOwan, “Facial expression recognition based on Local Binary Patterns: A comprehensive study”, Image and Vision Computing, pp. 803– 816, (2009).
  13. Aliaa A. A. Youssif, Wesam A. A. Asker “, Automatic Facial Expression Recognition System Based on Geometric and Appearance Features”, Canadian Center of Science and Education, Vol. 4, pp 115-124 (2011).
  14. SL Happy, A. Routray,” Automatic Facial Expression Recognition Using Features of Salient Facial Patches”, IEEE Transactions on Affective Computing, pp. 1-12, (2013).
  15. A. Poursaberi, A Noubari, M Gavrilova, and N Yanushkevich, ”Gauss–Laguerre wavelet textural feature fusion with geometrical information for facial expression Identification”, EURASIP Journal on Image and Video Processing Springer, pp: 1-13, (2012).
  16. J Chen, Shan, Chu, Zhao, Matti, Xilin, Wen, “WLD: A Robust Local Image Descriptor”, ieee transactions on pattern analysis and machine intelligence,pp. 1-15, (2009).
  17. Fatima Salmam, Mohamed KISSI, “Facial Expression Recognition using Decision Trees”,13th International Conference on Computer Graphics, Imaging, and Visualization IEEE, pp. 125–130, (2016).
  18. Sunil Kumar, M.K. Bhuyan, B. K. Chakraborty, ” Extraction of informative regions of a face for facial expression recognition”, IET Computer Vision Journals vol 10, pp. 567 – 576(2016).
  19. S. Katsigiannis, G. Keramidas, D. Maroulis, “Contourlet Transform for Texture Representation of Ultrasound Thyroid Images” 6th IFIP WG 12.5 International Conf., AIAI 2010 Springer, pp. 138–145.,(2010).
  20. Aysegu, Y. Demir, C. Guzelis, ”A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering”, Neural Comput&Applic Springer, pp. 131–142,(2016).
  21. Md. Uddin, W Khaksar, J Torresen, “Facial Expression Recognition Using Salient Features and Convolutional Neural Network”, a special section on visual surveillance and biometrics: practices, challenges, and possibilities IEEE Access Vol 5, pp: 26146 – 26161, (2017).
  22. M. Nazir, Z. Jan, M. Sajjad,” Facial expression recognition using histogram of oriented gradients based transformed features”, Cluster Comput DOI 10.1007/s10586-017-0921-5 Springer, pp: 1-10, (2017).
  23. Yi Ji, Khalid Idrissi,” Automatic facial expression recognition based on spatiotemporal descriptors”, Pattern Recognition Letters Elsevier, pp 1373-1380, (2012).
  24. M. Abdulrahman, Alaa Eleyan,”Facial Expression Recognition Using Support Vector Machines”, 23nd Signal Processing and Communications Appl. Conf. IEEE, pp.1-4( 2015).
  25. Md. Zia Uddin, J. J. Lee, T.-S. Kim,” An enhanced independent component-based human facial expression recognition from the video”, IEEE Transactions on Consumer Electronics, Vol. 55 IEEE Transactions, pp. 2216-2224,(2009).
  26. Byungyong Ryu, A Rivera, J. Kim, O. Chae” Local Directional Ternary Pattern for Facial Expression Recognition” IEEE Transactions on Image Processing, pp: 6006–6018, (2017).
  27. Shervin Minaee, AmiraliAbdolrashidi, “Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network” arXiv:1902.01019v1 [cs.CV] ,pp. 1-8,4 Feb (2019).
  28. Fengping An, Zhiwen Liu,” Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM”, International Journal of Computer Graphics Springer, pp.1-16, (2019).
S. Du, Y. Tao, A. M. Martinez, “Compound facial expressions of emotion”. PNAS, pp.1454–1462, (2014).

IInternational Journal of Scientific Research in Computer Science, Engineering and Information Technology

Downloads

Published

2021-08-26

Issue

Volume 8, Issue 5, September-October-2021

Section

Research Articles

License

Copyright (c) IJSRCSEIT

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Ratnalata Gupta, L. K. Vishwamitra, " The Comprehensive Study of Facial Expression Recognition on Video" International Journal of Scientific Research in Computer Science, Engineering and Information Technology(IJSRCSEIT), ISSN : 2456-3307, Volume 8, Issue 5, pp.01-08, September-October-2021.