Facial Expressions Detection and Recognition Using Neural Networks

Authors(2) :-Er. Navleen Kour, Dr. Naveen Kumar Gondhi

Facial Expressions are one of the most robust way of non verbal information exchange in day-to-day life. Changes occurring in emotions of a human being directly efficate the behavior of a person. In this progressive world, a numerous biometric have been evolved, each having its own purport. However, all these statistics play vital job to convey communication from one individual to another but 55% i.e a major part of communication transpire by facial expressions. Facial Expression recognition process concentrates on discerning the changes in expressions of facial muscles that automatically reflect switching of one’s mind from one state to another. Humans can recognize Expressions without any effort and almost instantaneously but that are not the case with a machine since its challenges are very dynamic like orientation, lightening, pose, facial expressions, etc. So, the process of wrenching out or extricating the facial feature points or landmarks is often very challenging. To recognize the fiducial points on the facial features and drawing out these points, that generally lie on eyes corners, chin, eyebrows, etc, facial landmarking is done. Our landmarking technique combines Viola-Jones detection algorithm for feature detection with Harris corner detection and then coarse to fine strategy is implemented using an efficient algorithm. Using the Haar like features reduces the cost of brute force search, also provides advantage of speed. Additional selection of sub-regions is also exploited using anthropometric constraints, to limit the search region. This further reduces false detection rate and improves accuracy significantly. A sub- algorithm named Iterative best fit algorithm is used find a land mark exploiting its commonality and geometric configuration and can be used in other contexts as well. This method is then tested on JAFEE database, Yale database, AT&T database and the database constructed using my own images named as Smart database and this method provides the satisfactory accuracy.

Authors and Affiliations

Er. Navleen Kour
Department of Computer Science Shri Mata Vaishno Devi University Katra, India
Dr. Naveen Kumar Gondhi
Department of Computer Science Shri Mata Vaishno Devi University Katra, India

Component, Biometrics, Facial Land Marking, Facial Emotions Recognition, Viola-Jones Algorithm, Harris Corner Detection.

  1. P. Ekman et al, "Facial Action Coding System Investigator’s Guide," A Human Face, Salt Lake City, UT, Consulting Psychological Press,2002.
  2. Ekman P et al, "The Facial Action Coding System A Technique for the Measurement of Facial Movement" , San Francisco ;Consulting Psychological Press, 1978.
  3. F Dornaika, F Davoine, ," Online appearance-based face and facial feature tracking" Washington, DC, USA, in Proc. Of Int. Conf .on Pattern Recognition, 2004, vol. 3. pp. 814–817
  4. J Cohn, A Zlochower, JJJ Lien, T Kanade.," Feature-point tracking by optical flow discriminates subtle differences in facial expression" , in Proc. Of IEEE Int. Conf. on Automatic Face and Gesture Recognition , Nara, Japan, 1998, pp. 396–401
  5. M Pantic, LJM Rothkrantz, "Automatic analysis of facial expressions: the state of the art" , IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1424–1445 (2000) 
  6. K Liu, A Weissenfeld, J Ostermann, X Luo," Robust AAM building for morphing in an image-based facial animation system" , in Proc .of Int. Conf. on Multimedia and Expo., Hannover, Germany, 2008, pp. 933–936
  7. S Ioannou, G Caridakis, K Karpouzis, S Kollias," Robust feature detection for facial expression recognition" , J. Image Video Process. 2007(2), 5–5 (2007)
  8. UPark,AKJain, "3D face reconstruction from stereo images ",in Proc. Of Int. Workshop on Video Processing for Security., Quebec City, Canada, 2006, p. 41
  9. AA Salah, N Alyüz, L Akarun," Registration of 3D face scans with average face models" . J. Electron. Imag. 17(1), 011006 (2008)
  10. N Pears, T Heseltine, M Romero, "From 3D point clouds to pose-normalised depth maps" . Int. J. Comput. Vis. 89(2), 152–176 (2010)
  11. C. Du, Q. Wu, J. Yang, and Z. Wu, "SVM based ASM for facial land- marks location," in Proc. IEEE International Conference on Computer and Information Technology (CIT’08), Nov 2008, pp. 321–326.
  12. T. F. Cootes and C. J. Taylor, "Active shape models - smart snakes," in Proc. British Machine Vision Conference (BMVC’92), Leeds, UK, set 1992, pp. 266–275
  13. T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active appearance models," in Proc. of the European Conference on Computer Vision (ECCV’98), Freiburg, DE, Jun 1998, pp. 484–498
  14. D. Vukadinovic and M. Pantic, "Fully automatic facial feature point detection using gabor feature based boosted classifiers," in Proc. IEEE International Conference on Systems, Man and Cybernetics (SCM’05), Waikoloa, Hawaii, Dec 2005, pp. 1692–1698.
  15. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. CVPR, 2001, vol. 1, pp. 511–518.
  16. J. Canny," A Computational Approach to Edge Detection" ,IEEE Trans. Pattern Analysis Machine Intelligence, vol. 8, no. 6, June 1986.
  17. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 1, pp. 39–58, Jan. 2009
  18. Essa, I.A.; Pentland, A.P., Coding, analysis, interpretation, and recognition of facial expressions, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume:19,Issue:7,July 1997, Page(s): 757 – 763
  19. Rosenblum, M.; Yacoob, Y.; Davis, L.S., "Human expression recognition from motion using a radial basis function network architecture" , IEEE Transactions on Neural Networks, Volume: 7 Issue: 5, Sept. 1996, Page(s): 1121 –1138
  20. G. Donato, M.S. Barlett, J.C. Hager, P. Ekman, T.J. Sejnowski, "Classifying facial actions" , IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 21, No 10,pp 974–989, 1999-2000
  21. Gizatdinova Y, Surakka V ," Feature-based detection of facial landmarks from neutral and expressive facial image" ,. IEEE Trans Pattern Anal Mach Intell 28:135139, 2006
  22. HC, Wu CY, Lin TM ," Facial Expression Recognition Using Image Processing Techniques and Neural Networks Advances in Intelligent Systems & Application" ,. Springer-Verlag Berlin Heidelberg, pp 259–267,2013.
  23. Ma L, Khorasani K ," Facial expression recognition using constructive feedforward neural networks" , IEEE Trans Syst Man Cybern B: Cybern 34:1588–1595,2004.
  24. Alessandro, L.F., "A neural network facial expression recognition system using unsupervised local processing" In: ISPA 2001. 2nd international symposium on image and signal processing and analysis, Pula, CROATIE, pp. 628–632 (2001)
  25. Saatci, Y., Town, C. "Cascaded classification of gender and facial expression using active appearance model" ,. In: FGR 2006. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, pp. 393–400. IEEE Computer Society Press, Washington, DC, USA (2006)
  26. Yacoob, Y., Davis, L.S." Recognizing human facial expressions from long image sequences using optical flow" IEEE Trans. Pattern Anal. Mach. Intell. 18(6), 636– 642 .
  27. M. A. Nicolaou, H. Gunes, and M. Pantic, "Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space," IEEE Transactions on Affective Computing, vol. 2, no. 2, pp. 92–105, 2011.
  28. H. A. Rowley, S. Baluja, and T. Kanade, "Neural network-based face detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 23–38, 1998.
  29. P. J. Phillips, H. Moon, P. J. Rauss, and S. A. Rizvi, "The FERET evaluation methodology for face-recognition algorithms," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090–1104, 2000.
  30. M.Lyons, J.Budynek, and S.Akamastu, "Automatic Classification of Single Facial Images" , IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.21, 1999, pp.1357- 1362.
  31. R.W. Picard, E. Vyzas, J. Healey, "Toward Machine Emotional Intelligence: Analysis of Affective Physiological State," in IEEE Transactions Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1175- 1191, 2001.
  32. K.H. Park, H.E. Lee, Y. Kim, Z.Z. Bien, "A Steward Robot for HumanFriendly Human-Machine Interaction in a Smart House Environment," in IEEE Transactions on Automation Science and Engineering, vol. 5, no. 1, pp. 21-25, 2008.
  33. C. Lisetti, S. Brown, K. Alvarez, A. Marpaung, "A Social Informatics Approach to Human-Robot Interaction with a Service Social Robot," in IEEE Systems, Men, and Cybernetics. Special Edition on Human-Robot Interaction, vol. 34 no. 2, 2004
  34. M. Pantic and L.J. Rothkrantz, "Automatic analysis of facial expressions: The state of the art," in IEEE Trans. Pattern Anal. Mach. Intell. vol. 22, no. 12, pp. 1424-1445, 2000
  35. Y. S. Gao, M. K. H. Leung, S. C. Hui, and M. W. Tananda, "Facial expression recognition from line-based caricature," IEEE Trans. System, Man, & Cybernetics (Part A), vol. 33, no. 3, pp. 407-412(May, 2003).
  36. L. Ma and K. Khorasani, "Facial expression recognition using constructive feedforward neural networks," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 3, pp. 1588–1595, Jun. 2004.

Publication Details

Published in : Volume 2 | Issue 7 | September 2017
Date of Publication : 2017-09-30
License:  This work is licensed under a Creative Commons Attribution 4.0 International License.
Page(s) : 197-209
Manuscript Number : CSEIT174425
Publisher : Technoscience Academy

ISSN : 2456-3307

Cite This Article :

Er. Navleen Kour, Dr. Naveen Kumar Gondhi, "Facial Expressions Detection and Recognition Using Neural Networks", International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN : 2456-3307, Volume 2, Issue 7, pp.197-209, September-2017. |          | BibTeX | RIS | CSV

Article Preview