Deception Detection Using Facial and Audio Transcript Features : A Review

Dr. Sheshang Degadwala; Radhika Thakkar

doi:10.32628/CSEIT2410584

Authors

Dr. Sheshang Degadwala Professor & Head, Department of Computer Engineering, Sigma University, Vadodara, Gujarat, India Author
Radhika Thakkar Research Scholar, Department of Computer Engineering, Sigma University, Vadodara, Gujarat, India Author

DOI:

https://doi.org/10.32628/CSEIT2410584

Keywords:

Deception Detection, Facial Features, Audio Transcripts, Machine Learning, Deep Learning, Multimodal Analysis, Feature Extraction

Abstract

Deception detection through facial and audio transcript features has gained traction due to its potential in enhancing security and communication integrity. This review aims to consolidate existing research on leveraging facial and audio features for identifying deceptive behavior. The motivation behind this study is the increasing demand for reliable deception detection mechanisms in various domains, including security and psychology. Despite advancements, limitations persist in achieving high accuracy across diverse contexts and individual differences. The objective of this review is to evaluate the effectiveness of different methods used in detecting deception from facial expressions and audio cues, identifying strengths and weaknesses of each approach, and suggesting future directions for improving accuracy through advanced techniques.

Downloads

Download data is not yet available.

References

Dinges, Laslo, et al. “Exploring Facial Cues: Automated Deception Detection Using Artificial Intelligence.” Neural Computing and Applications, vol. 0123456789, 2024, https://doi.org/10.1007/s00521-024-09811-x. DOI: https://doi.org/10.1007/s00521-024-09811-x

Prome, Shanjita Akter, et al. “Deception Detection Using Machine Learning (ML) and Deep Learning (DL) Techniques: A Systematic Review.” Natural Language Processing Journal, vol. 6, no. October 2023, 2024, p. 100057, https://doi.org/10.1016/j.nlp.2024.100057. DOI: https://doi.org/10.1016/j.nlp.2024.100057

Abdulridha, Fahad, and Baraa M. Albaker. “Non-Invasive Real-Time Multimodal Deception Detection Using Machine Learning and Parallel Computing Techniques.” Social Network Analysis and Mining, vol. 14, no. 1, 2024, https://doi.org/10.1007/s13278-024-01255-4. DOI: https://doi.org/10.1007/s13278-024-01255-4

Bahaa, Mohamed, et al. “Advancing Automated Deception Detection: A Multimodal Approach to Feature Extraction and Analysis.” Computer Vision and Pattern Recognition, July 2024, pp. 1–13, https://doi.org/10.48550/arXiv.2407.06005.

Guo, Xiaobao, et al. “Benchmarking Cross-Domain Audio-Visual Deception Detection.” IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2024, pp. 1–10, http://arxiv.org/abs/2405.06995.

D’Ulizia, Arianna, et al. “Detecting Deceptive Behaviours through Facial Cues from Videos: A Systematic Review.” Applied Sciences (Switzerland), vol. 13, no. 16, 2023, https://doi.org/10.3390/app13169188. DOI: https://doi.org/10.3390/app13169188

Yildirim, Suleyman, et al. “The Influence of Micro-Expressions on Deception Detection.” Multimedia Tools and Applications, vol. 82, no. 19, 2023, pp. 29115–33, https://doi.org/10.1007/s11042-023-14551-6. DOI: https://doi.org/10.1007/s11042-023-14551-6

Victor, Diaz, et al. “Enhancing Deception Detection with Exclusive Visual Features Using Deep Learning.” International Journal of Performability Engineering, vol. 19, no. 8, 2023, p. 547, https://doi.org/10.23940/ijpe.23.08.p7.547558. DOI: https://doi.org/10.23940/ijpe.23.08.p7.547558

Nam, Borum, et al. “FacialCueNet: Unmasking Deception - an Interpretable Model for Criminal Interrogation Using Facial Expressions.” Applied Intelligence, vol. 53, no. 22, 2023, pp. 27413–27, https://doi.org/10.1007/s10489-023-04968-9. DOI: https://doi.org/10.1007/s10489-023-04968-9

Mansbach, Noa, and Amos Azaria. “Meta Learning Based Deception Detection from Speech.” Applied Sciences (Switzerland), vol. 13, no. 1, 2023, https://doi.org/10.3390/app13010626. DOI: https://doi.org/10.3390/app13010626

Tsuchiya, Kento, et al. “Detecting Deception Using Machine Learning with Facial Expressions and Pulse Rate.” Artificial Life and Robotics, vol. 28, no. 3, 2023, pp. 509–19, https://doi.org/10.1007/s10015-023-00869-9. DOI: https://doi.org/10.1007/s10015-023-00869-9

Celniak, Weronika, et al. “Intelligent Eye-Tracker-Based Methods for Detection of Deception: A Survey.” Electronics (Switzerland), vol. 12, no. 22, 2023, https://doi.org/10.3390/electronics12224627. DOI: https://doi.org/10.3390/electronics12224627

Fernandes, Sinead V., and Muhammad Sana Ullah. “A Comprehensive Review on Features Extraction and Features Matching Techniques for Deception Detection.” IEEE Access, vol. 10, 2022, pp. 28233–46, https://doi.org/10.1109/ACCESS.2022.3157821. DOI: https://doi.org/10.1109/ACCESS.2022.3157821

Dong, Zizhao, et al. “Intentional-Deception Detection Based on Facial Muscle Movements in an Interactive Social Context.” Pattern Recognition Letters, vol. 164, 2022, pp. 30–39, https://doi.org/10.1016/j.patrec.2022.10.008. DOI: https://doi.org/10.1016/j.patrec.2022.10.008

Pérez-Rosas, Verónica, et al. “Deception Detection Using Real-Life Trial Data.” ICMI 2015 - Proceedings of the 2015 ACM International Conference on Multimodal Interaction, 2015, pp. 59–66, https://doi.org/10.1145/2818346.2820758. DOI: https://doi.org/10.1145/2818346.2820758

A. Kaur, A. Noori Hoshyar, V. Saikrishna, S. Firmin, and F. Xia, (2024). Deepfake video detection: challenges and opportunities, vol. 57, no. 6. Springer Netherlands. doi: 10.1007/s10462-024-10810-6. DOI: https://doi.org/10.1007/s10462-024-10810-6

A. Heidari, N. Jafari Navimipour, H. Dag, and M. Unal, (2024). Deepfake detection using deep learning methods: A systematic and comprehensive review, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 14, no. 2, pp. 1–45, doi: 10.1002/widm.1520. DOI: https://doi.org/10.1002/widm.1520

T. T. Nguyen et al., (2024). Deep learning for deepfakes creation and detection: A survey, Computer Vision and Image Understanding, vol. 223, no. 208070100, pp. 1–12, doi: 10.1016/j.cviu.2022.103525. DOI: https://doi.org/10.1016/j.cviu.2022.103525

A. Kaushal, A. Mina, A. Meena, and T. H. Babu, (2024). The societal impact of Deepfakes: Advances in Detection and Mitigation, 2023 14th International Conference on Computing Communication and Networking Technologies, ICCCNT pp. 1–7, doi: 10.1109/ICCCNT56998.2023.10307353. DOI: https://doi.org/10.1109/ICCCNT56998.2023.10307353

S. M. Abdullah et al., (2024). An Analysis of Recent Advances in Deepfake Image Detection in an Evolving Threat Landscape, arxiv, [Online]. Available: http://arxiv.org/abs/2404.16212

L. K. Seng, N. Mamat, H. Abas, N. Hamiza, and W. Ali (2024). AI Integrity Solutions for Deepfake Identification and Prevention, Open International Journal of Informatics (OIJI), vol. 12, no. 1, pp. 35–46. DOI: https://doi.org/10.11113/oiji2024.12n1.297

Degadwala, S., et al. “Improvements in Diagnosing Kawasaki Disease Using Machine Learning Algorithms.” 2024 4th International Conference on Pervasive Computing and Social Networking (ICPCSN), 2024, pp. 7–10, https://doi.org/10.1109/ICPCSN62568.2024.00009. DOI: https://doi.org/10.1109/ICPCSN62568.2024.00009

Mistry, S., and S. Degadwala. “Improved Multi-Type Vehicle Recognition with a Customized YOLO.” 2024 4th International Conference on Pervasive Computing and Social Networking (ICPCSN), 2024, pp. 361–65, https://doi.org/10.1109/ICPCSN62568.2024.00063. DOI: https://doi.org/10.1109/ICPCSN62568.2024.00063

Patel, V., and S. Degadwala. “Deployment of 3D-Conv-LSTM for Precipitation Nowcast via Satellite Data.” 2024 4th International Conference on Pervasive Computing and Social Networking (ICPCSN), 2024, pp. 984–88, https://doi.org/10.1109/ICPCSN62568.2024.00164. DOI: https://doi.org/10.1109/ICPCSN62568.2024.00164

Jagani, D., and S. Degadwala. “Monkeypox Skin Lesion Classification Using Fine-Tune CNN Model.” 2024 4th International Conference on Pervasive Computing and Social Networking (ICPCSN), 2024, pp. 37–41, https://doi.org/10.1109/ICPCSN62568.2024.00014. DOI: https://doi.org/10.1109/ICPCSN62568.2024.00014

Degadwala, Sheshang, et al. “DeepSpine: Multi-Class Spine X-Ray Conditions Classification Using Deep Learning.” Proceedings - 2024 3rd International Conference on Sentiment Analysis and Deep Learning, ICSADL 2024, 2024, pp. 8–13, https://doi.org/10.1109/ICSADL61749.2024.00008. DOI: https://doi.org/10.1109/ICSADL61749.2024.00008

Gadhiya, Niravkumar, et al. “Novel Approach for Data Encryption with Multilevel Compressive.” 7th International Conference on Inventive Computation Technologies, ICICT 2024, 2024, pp. 1368–72, https://doi.org/10.1109/ICICT60155.2024.10544502. DOI: https://doi.org/10.1109/ICICT60155.2024.10544502

Krishnamurthy, Vinay Nagarad Dasavandi, et al. “Predicting Hydrogen Fuel Cell Capacity Using Supervised Learning Models.” 7th International Conference on Inventive Computation Technologies, ICICT 2024, 2024, pp. 1934–38, https://doi.org/10.1109/ICICT60155.2024.10544401. DOI: https://doi.org/10.1109/ICICT60155.2024.10544401

Gadhiya, Niravkumar, et al. “A Review on Different Level Data Encryption through a Compression Techniques.” 7th International Conference on Inventive Computation Technologies, ICICT 2024, 2024, pp. 1378–81, https://doi.org/10.1109/ICICT60155.2024.10544803. DOI: https://doi.org/10.1109/ICICT60155.2024.10544803

Chakraborty, Utsho, et al. “Safeguarding Authenticity in Text with BERT-Powered Detection of AI-Generated Content.” 7th International Conference on Inventive Computation Technologies, ICICT 2024, 2024, pp. 34–37, https://doi.org/10.1109/ICICT60155.2024.10544590. DOI: https://doi.org/10.1109/ICICT60155.2024.10544590

Prajapati, Piyush M., et al. “Exploring Methods of Mitigation against DDoS Attack in an IoT Network.” 7th International Conference on Inventive Computation Technologies, ICICT 2024, 2024, pp. 1373–77, https://doi.org/10.1109/ICICT60155.2024.10544424. DOI: https://doi.org/10.1109/ICICT60155.2024.10544424

Agarwal, Ruhi Himanshu, et al. “Predictive Modeling for Thyroid Disease Diagnosis Using Machine Learning.” 7th International Conference on Inventive Computation Technologies, ICICT 2024, 2024, pp. 227–31, https://doi.org/10.1109/ICICT60155.2024.10544462. DOI: https://doi.org/10.1109/ICICT60155.2024.10544462

Soni, Deepika, et al. “Veterinary Medical Records Application Using AWS.” Proceedings - 2024 5th International Conference on Mobile Computing and Sustainable Informatics, ICMCSI 2024, 2024, pp. 578–84, https://doi.org/10.1109/ICMCSI61536.2024.00091. DOI: https://doi.org/10.1109/ICMCSI61536.2024.00091

Degadwala, Sheshang, et al. “Unveiling Cholera Patterns through Machine Learning Regression for Precise Forecasting.” Proceedings - 2024 5th International Conference on Mobile Computing and Sustainable Informatics, ICMCSI 2024, 2024, pp. 39–44, https://doi.org/10.1109/ICMCSI61536.2024.00012. DOI: https://doi.org/10.1109/ICMCSI61536.2024.00012

Pandya, D. D., et al. “Retraction: Diagnostic Criteria for Depression Based on Both Static and Dynamic Visual Features (IDCIoT 2023 - International Conference on Intelligent Data Communication Technologies and Internet of Things, Proceedings (2023) DOI: 10.1109/IDCIoT56793.2023.10053450).” IDCIoT 2023 - International Conference on Intelligent Data Communication Technologies and Internet of Things, Proceedings, 2023, p. 1, https://doi.org/10.1109/IDCIoT56793.2023.10554339. DOI: https://doi.org/10.1109/IDCIoT56793.2023.10053450

Mewada, Shubbh, et al. “Improved CAD Classification with Ensemble Classifier and Attribute Elimination.” Proceedings - 2023 3rd International Conference on Ubiquitous Computing and Intelligent Information Systems, ICUIS 2023, 2023, pp. 238–43, https://doi.org/10.1109/ICUIS60567.2023.00048. DOI: https://doi.org/10.1109/ICUIS60567.2023.00048

Pandya, Darshanaben D., et al. “Advancements in Multiple Sclerosis Disease Classification Through Machine Learning.” Proceedings - 2023 3rd International Conference on Ubiquitous Computing and Intelligent Information Systems, ICUIS 2023, 2023, pp. 64–69, https://doi.org/10.1109/ICUIS60567.2023.00019. DOI: https://doi.org/10.1109/ICUIS60567.2023.00019

Degadwala, Sheshang, et al. “Enhancing Fleet Management with ESP8266-Based IoT Sensors for Weight and Location Tracking.” 3rd International Conference on Innovative Mechanisms for Industry Applications, ICIMIA 2023 - Proceedings, 2023, pp. 13–17, https://doi.org/10.1109/ICIMIA60377.2023.10425949. DOI: https://doi.org/10.1109/ICIMIA60377.2023.10425949

Degadwala, Sheshang, et al. “Enhancing Mesothelioma Cancer Diagnosis through Ensemble Learning Techniques.” 3rd International Conference on Innovative Mechanisms for Industry Applications, ICIMIA 2023 - Proceedings, 2023, pp. 628–32, https://doi.org/10.1109/ICIMIA60377.2023.10425887. DOI: https://doi.org/10.1109/ICIMIA60377.2023.10425887

Degadwala, Sheshang, et al. “Methods of Transfer Learning for Multiclass Hair Disease Categorization.” 2nd International Conference on Automation, Computing and Renewable Systems, ICACRS 2023 - Proceedings, 2023, pp. 612–16, https://doi.org/10.1109/ICACRS58579.2023.10404492. DOI: https://doi.org/10.1109/ICACRS58579.2023.10404492

Degadwala, Sheshang, et al. “DeepTread: Exploring Transfer Learning in Tyre Quality Classification.” International Conference on Sustainable Communication Networks and Application, ICSCNA 2023 - Proceedings, 2023, pp. 1448–53, https://doi.org/10.1109/ICSCNA58489.2023.10370168. DOI: https://doi.org/10.1109/ICSCNA58489.2023.10370168

Mewada, Shubbh, et al. “Enhancing Raga Identification in Indian Classical Music with FCN-Based Models.” International Conference on Sustainable Communication Networks and Application, ICSCNA 2023 - Proceedings, 2023, pp. 980–85, https://doi.org/10.1109/ICSCNA58489.2023.10370046. DOI: https://doi.org/10.1109/ICSCNA58489.2023.10370046

Degadwala, Sheshang, et al. “Revolutionizing Hops Plant Disease Classification: Harnessing the Power of Transfer Learning.” International Conference on Sustainable Communication Networks and Application, ICSCNA 2023 - Proceedings, 2023, pp. 1706–11, https://doi.org/10.1109/ICSCNA58489.2023.10370692. DOI: https://doi.org/10.1109/ICSCNA58489.2023.10370692

Degadwala, Sheshang, et al. “Crime Pattern Analysis and Prediction Using Regression Models.” International Conference on Self Sustainable Artificial Intelligence Systems, ICSSAS 2023 - Proceedings, 2023, pp. 771–76, https://doi.org/10.1109/ICSSAS57918.2023.10331747. DOI: https://doi.org/10.1109/ICSSAS57918.2023.10331747

Bhavesh Kataria Dr. Harikrishna B. Jethva, " Review of Advances in Digital Recognition of Indian Language Manuscripts, International Journal of Scientific Research in Science, Engineering and Technology, Print ISSN : 2395-1990, Online ISSN : 2394-4099, Volume 4, Issue 1, pp.1302-1318, January-February-2018. Available at doi : https://doi.org/10.32628/ijsrset1841215 DOI: https://doi.org/10.32628/IJSRSET1841215

Prajapati, Rohit, et al. “QoS Based Virtual Machine Consolidation for Energy Efficient and Economic Utilization of Cloud Resources.” International Conference on Self Sustainable Artificial Intelligence Systems, ICSSAS 2023 - Proceedings, 2023, pp. 951–57, https://doi.org/10.1109/ICSSAS57918.2023.10331674. DOI: https://doi.org/10.1109/ICSSAS57918.2023.10331674

Patel, Fagun, et al. “Recognition of Pistachio Species with Transfer Learning Models.” International Conference on Self Sustainable Artificial Intelligence Systems, ICSSAS 2023 - Proceedings, 2023, pp. 250–55, https://doi.org/10.1109/ICSSAS57918.2023.10331907. DOI: https://doi.org/10.1109/ICSSAS57918.2023.10331907

Patel, Fagun, et al. “Exploring Transfer Learning Models for Multi-Class Classification of Infected Date Palm Leaves.” International Conference on Self Sustainable Artificial Intelligence Systems, ICSSAS 2023 - Proceedings, 2023, pp. 307–12, https://doi.org/10.1109/ICSSAS57918.2023.10331746. DOI: https://doi.org/10.1109/ICSSAS57918.2023.10331746

Pandya, Darshanaben D., et al. “Advancing Erythemato-Squamous Disease Classification with Multi-Class Machine Learning.” 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud), I-SMAC 2023 - Proceedings, 2023, pp. 542–47, https://doi.org/10.1109/I-SMAC58438.2023.10290599. DOI: https://doi.org/10.1109/I-SMAC58438.2023.10290599

Degadwala, Sheshang, et al. “Determine the Degree of Malignancy in Breast Cancer Using Machine Learning.” 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud), I-SMAC 2023 - Proceedings, 2023, pp. 483–87, https://doi.org/10.1109/I-SMAC58438.2023.10290430. DOI: https://doi.org/10.1109/I-SMAC58438.2023.10290430

Pandya, Darshanaben D., et al. “Unveiling the Power of Collective Intelligence: A Voting-Based Approach for Dementia Classification.” 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud), I-SMAC 2023 - Proceedings, 2023, pp. 478–82, https://doi.org/10.1109/I-SMAC58438.2023.10290165. DOI: https://doi.org/10.1109/I-SMAC58438.2023.10290165

Bhavesh Kataria, Dr. Harikrishna B. Jethva (2021). Optical Character Recognition of Sanskrit Manuscripts Using Convolution Neural Networks, Webology, ISSN: 1735-188X, Volume 18 Issue 5, October-2021, pp. 403-424. Available at https://www.webology.org/abstract.php?id=1681

Patel, Ankur, et al. “Enhancing Traffic Management with YOLOv5-Based Ambulance Tracking System.” Canadian Conference on Electrical and Computer Engineering, vol. 2023-September, 2023, pp. 528–32, https://doi.org/10.1109/CCECE58730.2023.10288751. DOI: https://doi.org/10.1109/CCECE58730.2023.10288751

Degadwala, Sheshang, et al. “Revolutionizing Prostate Cancer Diagnosis: Harnessing the Potential of Transfer Learning for MRI-Based Classification.” Proceedings of the 4th International Conference on Smart Electronics and Communication, ICOSEC 2023, 2023, pp. 938–43, https://doi.org/10.1109/ICOSEC58147.2023.10275879. DOI: https://doi.org/10.1109/ICOSEC58147.2023.10275879

Patel, Krunal, et al. “Safety Helmet Detection Using YOLO V8.” Proceedings - 2023 3rd International Conference on Pervasive Computing and Social Networking, ICPCSN 2023, 2023, pp. 22–26, https://doi.org/10.1109/ICPCSN58827.2023.00012. DOI: https://doi.org/10.1109/ICPCSN58827.2023.00012

Mehta, Jay N., et al. “EEG Brainwave Data Classification of a Confused Student Using Moving Average Feature.” Proceedings - 2023 3rd International Conference on Pervasive Computing and Social Networking, ICPCSN 2023, 2023, pp. 1461–66, https://doi.org/10.1109/ICPCSN58827.2023.00243. DOI: https://doi.org/10.1109/ICPCSN58827.2023.00243

Bhavesh Kataria, Dr. Harikrishna B. Jethva (2020). Sanskrit Character Recognition using Convolutional Neural Networks : A Survey. International Journal of Advanced Science and Technology, 29(7), 1059 – 1071, May 2020. Retrieved from http://sersc.org/journals/index.php/IJAST/article/view/15068

Pareek, Naveen Kumar, et al. “Prediction of CKD Using Expert System Fuzzy Logic & AI.” Proceedings of the 2023 2nd International Conference on Augmented Intelligence and Sustainable Systems, ICAISS 2023, 2023, pp. 103–08, https://doi.org/10.1109/ICAISS58487.2023.10250477. DOI: https://doi.org/10.1109/ICAISS58487.2023.10250477

Degadwala, Sheshang, et al. “Enhancing Prostate Cancer Diagnosis: Leveraging XGBoost for Accurate Classification.” Proceedings of the 2023 2nd International Conference on Augmented Intelligence and Sustainable Systems, ICAISS 2023, 2023, pp. 1776–81, https://doi.org/10.1109/ICAISS58487.2023.10250511. DOI: https://doi.org/10.1109/ICAISS58487.2023.10250511

Degadwala, Sheshang, et al. “Empowering Maxillofacial Diagnosis Through Transfer Learning Models.” Proceedings of the 5th International Conference on Inventive Research in Computing Applications, ICIRCA 2023, 2023, pp. 728–32, https://doi.org/10.1109/ICIRCA57980.2023.10220830. DOI: https://doi.org/10.1109/ICIRCA57980.2023.10220830

Degadwala, Sheshang, et al. “Enhancing Alzheimer Stage Classification of MRI Images through Transfer Learning.” Proceedings of the 5th International Conference on Inventive Research in Computing Applications, ICIRCA 2023, 2023, pp. 733–37, https://doi.org/10.1109/ICIRCA57980.2023.10220651. DOI: https://doi.org/10.1109/ICIRCA57980.2023.10220651

Degadwala, Sheshang, et al. “Optimizing Hindi Paragraph Summarization through PageRank Method.” Proceedings of the 2nd International Conference on Edge Computing and Applications, ICECAA 2023, 2023, pp. 504–09, https://doi.org/10.1109/ICECAA58104.2023.10212107. DOI: https://doi.org/10.1109/ICECAA58104.2023.10212107

Dasavandi Krishnamurthy, Vinay Nagarad, et al. “Forecasting Future Sea Level Rise: A Data-Driven Approach Using Climate Analysis.” Proceedings of the 2nd International Conference on Edge Computing and Applications, ICECAA 2023, 2023, pp. 646–51, https://doi.org/10.1109/ICECAA58104.2023.10212399. DOI: https://doi.org/10.1109/ICECAA58104.2023.10212399

Degadwala, Sheshang, et al. “Cancer Death Cases Forecasting Using Supervised Machine Learning.” 2023 4th International Conference on Electronics and Sustainable Communication Systems, ICESC 2023 - Proceedings, 2023, pp. 903–07, https://doi.org/10.1109/ICESC57686.2023.10193685. DOI: https://doi.org/10.1109/ICESC57686.2023.10193685