Facial Expression Classification in The Wild by Traditional Machine Learning and Deep Learning
DOI:
https://doi.org/10.52783/jns.v14.3158Keywords:
Facial expression classification, computer vision, deep learning, CNN, wild datasetAbstract
Analyzing facial expressions is a fundamental aspect of computer vision, with significant importance in applications such as human-machine communication and interaction. Recognizing expressions in uncontrolled environments (wild), with varied lighting, poses, and occlusions, is particularly challenging. This study compares traditional methods (PCA, LBP, LDA, and HOG) with deep learning technique (CNN) using CK+ (posed images) and FER2013 (candid images) datasets. Results show that CNN outperforms traditional methods, achieving 34% higher accuracy in uncontrolled conditions. While traditional approaches excel in controlled settings, deep learning proves more effective overall for FER in natural environments
Downloads
Metrics
References
Nisha V, Shilpa T., Facial Emotion Recognition using Machine Learning Techniques, IOSR Journal of Computer Engineering (IOSR-JCE), Volume 23, Issue 5, Ser. I (Sep. –Oct. 2021), PP 41-49.
Tsalera, E.; Papadakis, A.; Samarakou, M.; Voyiatzis, I. Feature Extraction with Handcrafted Methods and Convolutional Neural Networks for Facial Emotion Recognition. Appl. Sci. 2022, 12, 8455.
Htay, Moe., Feature extraction and classification methods of facial expression: a survey. Computer Science and Information Technologies. 2. 2021,26-32.
Wafa Mellouk, Wahida Handouzi, Facial emotion recognition using deep learning: review and insights, Procedia Computer Science, Volume 175, 2020, Pages 689-694.
Debnath, T., Reza, M.M., Rahman, A. et al. Four-layer ConvNet to facial emotion recognition with minimal epochs and the significance of data diversity. Sci Rep 12, 6991 (2022).
Khan, Amjad Rehman., Facial Emotion Recognition Using Conventional Machine Learning and Deep Learning Methods: Current Achievements, Analysis and Remaining Challenges. Information. 13. 2022, 268.
S. Li and W. Deng, "Deep Facial Expression Recognition: A Survey," in IEEE Transactions on Affective Computing, vol. 13, no. 3, pp. 1195-1215, 1 July-Sept. 2022.
O. S. Ekundayo and S. Viriri, "Facial Expression Recognition: A Review of Trends and Techniques," in IEEE Access, vol. 9, pp. 136944-136973, 2021.
Ekaterina Ivanova, Georgii Borzunov, Optimization of machine learning algorithm of emotion recognition in terms of human facial expressions, Procedia Computer Science, Volume 169,2020, Pages 244-248.
Liu, Zhenhai & Wang, Hanzi & Yan, Yan & Guo, Guanjun., Effective Facial Expression Recognition via the Boosted Convolutional Neural Network. 2015, 179-188.
A. Hewa,0. Nomir, A. Saleh, Facial Expression Recognition Based on Principal Components Analysis, International Journal of Intelligent Computing and Information Science, Vo1.16 No. 4 October 2016.
Varma, Satishkumar, Megha Shinde, and Satishkumar S. Chavan. "Analysis of PCA and LDA features for facial expression recognition using SVM and HMM classifiers." In Techno-Societal 2018: Proceedings of the 2nd International Conference on Advanced Technologies for Societal Applications-Volume 1, pp. 109-119. Springer International Publishing, 2020.
Zafar, Sahar & Ali, Fayyaz & Guriro, Subhash & Ali, Irfan & Khan, Asif & Zaidi, Adnan. (2019). Facial Expression Recognition with Histogram of Oriented Gradients using CNN. Indian Journal of Science and Technology. 12. 2019.
Arora Malika; Kumar, Munish, AutoFER: PCA and PSO based automatic facial emotion recognition Multimedia Tools and Applications; Dordrecht Vol. 80, Iss. 2, (Jan 2021): 3039-3049.
Kola, Durga & Samayamantula, Srinivas., A novel approach for facial expression recognition using local binary pattern with adaptive window. Multimedia Tools and Applications. 80. 2021,1-20.
Ben Niu, Zhenxing Gao, Bingbing Guo, "Facial Expression Recognition with LBP and ORB Features", Computational Intelligence and Neuroscience, vol. 2021, Article ID 8828245, 10 pages, 2021.
D. Lakshmi, R. Ponnusamy, Facial emotion recognition using modified HOG and LBP features with deep stacked autoencoders, Microprocessors and Microsystems, Volume 82,2021,103834, ISSN 0141-9331.
R. Ravi, S. V. Yadhukrishna and R. Prithvi raj, "A Face Expression Recognition Using CNN & LBP," 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, 2020, pp. 684-689.
www.kaggle.com, accessed on. 23.02.2024.
Chaudhari, Aayushi & Bhatt, Chintan & Adiraju, Achyut & Mazzeo, Pier Luigi. (2022). ViTFER: Facial Emotion Recognition with Vision Transformers. Applied System Innovation. 5. 80. 10.3390/asi5040080.
G. K. Sahoo, J. Ponduru, S. K. Das and P. Singh, "Deep Leaning-Based Facial Expression Recognition in FER2013 Database: An in-Vehicle Application," 2022 IEEE 19th India Council International Conference (INDICON), Kochi, India, 2022, pp. 1-6, doi: 10.1109/INDICON56171.2022.10040121.
Neha A. Chinchanikar, Facial Expression Recognition Using Deep Learning: A Review, International Research Journal of Engineering and Technology (IRJET), Volume: 06 Issue: 06 | June 2019.
C. Mejia-Escobar, M. Cazorla and E. Martinez-Martin, "Improving Facial Expression Recognition Through Data Preparation and Merging," in IEEE Access, vol. 11, pp. 71339-71360, 2023, doi: 10.1109/ACCESS.2023.3293728.
Khaireddin, Yousif, and Zhuofa Chen. "Facial emotion recognition: State of the art performance on FER2013." arXiv preprint arXiv:2105.03588 (2021).
Huang Y, Chen F, Lv S, Wang X. Facial Expression Recognition: A Survey. Symmetry. 2019; 11(10):1189. https://doi.org/10.3390/sym11101189.
Kim JH, Poulose A, Han DS. The Extensive Usage of the Facial Image Threshing Machine for Facial Emotion Recognition Performance. Sensors. 2021; 21(6):2026. https://doi.org/10.3390/s21062026
Mejia-Escobar C, Cazorla M, Martinez-Martin E. Towards a Better Performance in Facial Expression Recognition: A Data-Centric Approach. Comput Intell Neurosci. 2023 Nov 3;2023:1394882. doi: 10.1155/2023/1394882. PMID: 37954097; PMCID: PMC10637848.
Barrett LF, Adolphs R, Marsella S, Martinez AM, Pollak SD. Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychol Sci Public Interest. 2019 Jul;20(1):1-68. doi: 10.1177/1529100619832930. Erratum in: Psychol Sci Public Interest. 2019 Dec;20(3):165-166. doi: 10.1177/1529100619889954. PMID: 31313636; PMCID: PMC6640856.
N. Mishra and A. Bhatt, "Feature Extraction Techniques in Facial Expression Recognition," 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 2021, pp. 1247-1251, doi: 10.1109/ICICCS51141.2021.9432192.
S. Li and W. Deng, "A Deeper Look at Facial Expression Dataset Bias," in IEEE Transactions on Affective Computing, vol. 13, no. 2, pp. 881-893, 1 April-June 2022, doi: 10.1109/TAFFC.2020.2973158.
Wafi, Muhammad & Bachtiar, Fitra & Utaminingrum, Fitri. (2023). Feature extraction comparison for facial expression recognition using adaptive extreme learning machine. International Journal of Electrical and Computer Engineering (IJECE). 13. 2023, pp1113-1122.
T. Ojala, M. Pietikainen, and D. Harwood, “A comparative ¨ study of texture measures with classification based on featured distributions,” Pattern Recognition, vol. 29, no. 1, pp. 51–59, 1996.
I.Michael Revina, W.R. Sam Emmanuel, A Survey on Human Face Expression Recognition Techniques, JOURNAL OF KING SAUD UNIVERSITY: COMPUTER AND INFORMATION SCIENCES (JUL 2021), Vol. 33, no. 6, pp. 619 – 628.
S. Liao, M. W. K. Law, and A. C. S. Chung, “Dominant local binary patterns for texture classification,” IEEE Trans. Image Process., vol. 18, no. 5, pp. 1107–1118, May 2009. [crossref]
K. Mistry, L. Zhang, S. C. Neoh, C. P. Lim and B. Fielding, "A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition," in IEEE Transactions on Cybernetics, vol. 47, no. 6, pp. 1496-1509, June 2017.
Shan, Caifeng & Gong, Shaogang & Mcowan, Peter. (2009). Facial expression recognition based on Local Binary Patterns: A comprehensive study. Image and Vision Computing. 27.2009. 803-816.
Zixiang Fei, Erfu Yang, David Day-Uei Li, Stephen Butler, Winifred Ijomah & Huiyu Zhou. A survey on computer vision techniques for detecting facial features towards the early diagnosis of mild cognitive impairment in the elderly, Systems Science & Control Engineering, 7:1, 2019. 252-263.
Lei Zhao, Zengcai Wang, Guoxin Zhang, "Facial Expression Recognition from Video Sequences Based on Spatial-Temporal Motion Local Binary Pattern and Gabor Multiorientation Fusion Histogram", Mathematical Problems in Engineering, vol. 2017, Article ID 7206041, 12 pages, 2017.
Zhu, Dimin & Fu, Yuxi & Zhao, Xinjie & Wang, Xin & Yi, Hanxi. (2022). Facial Emotion Recognition Using a Novel Fusion of Convolutional Neural Network and Local Binary Pattern in Crime Investigation. Computational Intelligence and Neuroscience. 2022. 1-14.
S. S. Meher and P. Maben, "Face recognition and facial expression identification using PCA," 2014 IEEE International Advance Computing Conference (IACC), Gurgaon, India, 2014, pp. 1093-1098.
Saket Karve, Vasisht Shende, Rizwan Ahmed, A comparative analysis of feature extraction techniques for face recognition, 2018 International Conference on Communication, Information & Computing Technology (ICCICT), Feb. 2-3, Mumbai, India.
Sahoo, Ramesh & Pradhan, Sateesh. (2019). A Comparative Study on Hopfield Network with LBP, PCA and LDA for Face Recognition in Distorted Face Images. 102-110.
Bhattacharyya, Suman & Rahul, Kumar. Face recognition by linear discriminant analysis. International Journal of Communication Network Security. 2. 2013. 31-35.
Shih, Frank Y., Chao-Fa Chuang, and Patrick SP Wang. "Performance comparisons of facial expression recognition in JAFFE database." International Journal of Pattern Recognition and Artificial Intelligence 22.03 (2008): 445-459.
Greche, L., Akil, M., Kachouri, R. et al. A new pipeline for the recognition of universal expressions of multiple faces in a video sequence. J Real-Time Image Proc 17, 1389–1402 (2020).
Carcagnì, P., Del Coco, M., Leo, M.: Distante, Cosimo: Facial expression recognition and histograms of oriented gradients: a comprehensive study. SpringerPlus 4(1), 645 (2015).
Jeniffer Xin-Ying Lek, Jason Teo, "Academic Emotion Classification Using FER: A Systematic Review", Human Behavior and Emerging Technologies, vol. 2023, Article ID 9790005, 27 pages, 2023.
AKSOY, Orhan & GÜNEY, Selda. Sentiment Analysis from Face Expressions Based on Image Processing Using Deep Learning Methods. Journal of Advanced Research in Natural and Applied Sciences. 8. 2022.
Çayir, Aykut, Işil Yenidoğan, and Hasan Dağ. "Feature extraction based on deep learning for some traditional machine learning methods." 2018 3rd International conference on computer science and engineering (UBMK). IEEE, 2018, pp. 494-497.
Anil Audumbar Pise, Mejdal A. Alqahtani, Priti Verma, Purushothama K, Dimitrios A. Karras, Prathibha S, Awal Halifa, "Methods for Facial Expression Recognition with Applications in Challenging Situations", Computational Intelligence and Neuroscience, vol. 2022, Article ID 9261438, 17 pages, 2022.
Podder T, Bhattacharya D, Majumder P, Balas VE. 2023. A feature boosted deep learning method for automatic facial expression recognition. PeerJ Comput. Sci. 9:e1216.
Nur Alia Syahirah Badrulhisham and Nur Nabilah Abu Mangshor 2021 J. Phys.: Conf. Ser. 1962 012040.
O. S. Ekundayo and S. Viriri, "Facial Expression Recognition: A Review of Trends and Techniques," in IEEE Access, vol. 9, pp. 136944-136973, 2021, doi: 10.1109/ACCESS.2021.3113464.
Jyoti Kumari, R. Rajesh, K.M. Pooja, Facial Expression Recognition: A Survey, Procedia Computer Science,Volume 58,2015,Pages 486-491,ISSN 1877-0509
Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaría J, Fadhel MA, Al-Amidie M, Farhan L. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data. 2021;8(1):53. doi: 10.1186/s40537-021-00444-8. Epub 2021 Mar 31. PMID: 33816053; PMCID: PMC8010506.
Taye MM. Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions. Computers. 2023; 12(5):91. https://doi.org/10.3390/computers12050091.
Walecki, R., Rudovic, O., Schuller, B., Pavlovic, V., & Pantic, M. (2017). Deep Structured Learning for Facial Expression Intensity Estimation.
Browne, Matthew & Ghidary, Saeed & Mayer, Norbert. (2007). Convolutional Neural Networks for Image Processing with Applications in Mobile Robotics. 10.1007/978-3-540-75398-8_15.
Canedo D, Neves AJR. Facial Expression Recognition Using Computer Vision: A Systematic Review. Applied Sciences. 2019; 9(21):4678. https://doi.org/10.3390/app9214678.
André Teixeira Lopes a, Edilson de Aguiar b, Alberto F. De Souza a, Thiago Oliveira-Santos, “Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order”, Elsevier, Pattern Recognition 61 (2017) 610–628.
D. Venkataraman and N. S. Parameswaran, “Extraction of Facial Features for Depression Detection among Students”, International journal of pure and applied mathematics,2018, vol.118.
Caifeng Shan, Shaogang Gong and P. W. McOwan, "Robust facial expression recognition using local binary patterns," IEEE International Conference on Image Processing 2005, 2005, pp. II-370.
S. Shakya, S. Sharma and A. Basnet, "Human behavior prediction using facial expression analysis," 2016 International Conference on Computing, Communication and Automation (ICCCA), 2016, pp. 399-404.
Zixiang Fei, Erfu Yang, David Day-Uei Li, Stephen Butler, Winifred Ijomah & Huiyu Zhou (2019) A survey on computer vision techniques for detecting facial features towards the early diagnosis of mild cognitive impairment in the elderly, Systems Science & Control Engineering, 7:1, 252-263.
F. Zhang, T. Zhang, Q. Mao and C. Xu, "Geometry Guided Pose-Invariant Facial Expression Recognition," in IEEE Transactions on Image Processing, vol. 29, pp. 4445-4460, 2020, doi: 10.1109/TIP.2020.2972114.
S. Li and W. Deng, "Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition," in IEEE Transactions on Image Processing, vol. 28, no. 1, pp. 356-370, Jan. 2019, doi: 10.1109/TIP.2018.2868382.
O’Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G. V., Krpalkova, L., ... & Walsh, J. (2020). Deep learning vs. traditional computer vision. In Advances in Computer Vision: Proceedings of the 2019 Computer Vision Conference (CVC), Volume 1 1 (pp. 128-144). Springer International Publishing.
Karypidis, E., Mouslech, S. G., Skoulariki, K., & Gazis, A. (2022). Comparison Analysis of Traditional Machine Learning and Deep Learning Techniques for Data and Image Classification. arXiv preprint arXiv:2204.05983.
Georgescu, Mariana & Ionescu, Radu Tudor & Popescu, Marius. (2019). Local Learning With Deep and Handcrafted Features for Facial Expression Recognition. IEEE Access. PP. 1-1. 10.1109/ACCESS.2019.2917266.
C. Quan, Y. Qian, and F. Ren, Dynamic facial expression recognition based on K-order emotional intensity model, in Proc. IEEE Int. Conf. Robot. Biomimetics (ROBIO), Dec. 2014, pp. 11641168.
Mehta D, Siddiqui MFH, Javaid AY. Facial Emotion Recognition: A Survey and Real-World User Experiences in Mixed Reality. Sensors (Basel). 2018 Feb 1;18(2):416. doi: 10.3390/s18020416. PMID: 29389845; PMCID: PMC5856132.
Bansal, Sonal & Rustagi, Aditya & Kumar, Anupam. (2021). Alzheimer's disease diagnosis based on feature extraction using optimised crow search algorithm and deep learning. International Journal of Computer Applications in Technology. 65. 325. 10.1504/IJCAT.2021.117272.
Meena, Lakshminarayanan & Thambusamy, Velmurugan. (2023). Optimizing Facial Expression Recognition through Effective Preprocessing Techniques. Journal of Computer and Communications. 11. 86-101. 10.4236/jcc.2023.1112006.
Y. Lin et al., "Large-scale image classification: Fast feature extraction and SVM training," CVPR 2011, Colorado Springs, CO, USA, 2011, pp. 1689-1696, doi: 10.1109/CVPR.2011.5995477.
Krichen M. Convolutional Neural Networks: A Survey. Computers. 2023; 12(8):151. https://doi.org/10.3390/computers12080151.
Jubair F, Al-Karadsheh O, Malamos D, Al Mahdi S, Saad Y, Hassona Y. A novel lightweight deep convolutional neural network for early detection of oral cancer. Oral Dis. 2022 May;28(4):1123-1130. doi: 10.1111/odi.13825. Epub 2021 Mar 5. PMID: 33636041.
Yunfei Lai, A Comparison of Traditional Machine Learning and Deep Learning in Image Recognition 2019 J. Phys.: Conf. Ser. 1314 012148
M M, A M. Facial geometric feature extraction based emotional expression classification using machine learning algorithms. PLoS One. 2021 Feb 18;16(2): e0247131. doi: 10.1371/journal.pone.0247131. PMID: 33600467; PMCID: PMC7891769.
Chaudhari, Aayushi & Bhatt, Chintan & Adiraju, Achyut & Mazzeo, Pier Luigi. (2022). ViTFER: Facial Emotion Recognition with Vision Transformers. Applied System Innovation. 5. 80. 10.3390/asi5040080.
Garcia-Garcia, Jose & Penichet, Victor & Lozano, María. Emotion detection: a technology review. 2017. 1-8.
Caroppo, A., Leone, A. & Siciliano, P. Comparison Between Deep Learning Models and Traditional Machine Learning Approaches for Facial Expression Recognition in Ageing Adults. J. Comput. Sci. Technol. 35, 1127–1146 (2020). https://doi.org/10.1007/s11390-020-9665-4.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
Terms:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
