Performance Evaluation of Complex Face Emotion Recognition Using Convolution Neural Networks Under Diverse Conditions
Received: 20 August 2025 | Revised: 3 November 2025 and 27 November 2025 | Accepted: 28 November 2025 | Online: 9 February 2026
Corresponding author: Milind Talele
Abstract
Facial emotion recognition methods are computer vision-based algorithms for identifying facial emotions from an input image or video, with several applications in surveillance systems, healthcare, education, and retail. A complex facial emotion can be visualized as a combination of two or more basic emotions. This study presents a model for recognizing complex facial emotions using a convolutional neural network, trained on the FER2013 dataset and tested on the CEED dataset. This study investigates the performance of the model in variations, such as illumination conditions, demographic diversity, and image characteristics. Systematic and controlled experiments were conducted to validate the performance of the model and calculate the Complex Emotion Scores (CES). The average CES remained at 80% for dark images with an accuracy of approximately 89%. Female images tend to maintain slightly higher CES scores in light conditions than male ones. However, CES remained consistent on image file types, and the CPU utilization was found to be high for certain image formats.
Keywords:
Face Emotion Recognition (FER), complex face emotion recognition, Convolutional Neural Network (CNN), performance, CFER scoreDownloads
References
R. Plutchik, ''The Nature of Emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice,'' American Scientist, vol. 89, no. 4, 2001, Art. no. 344. DOI: https://doi.org/10.1511/2001.4.344
A. M. Abdulazeez and Z. S. Ageed, ''Face Emotion Recognition Based on Machine Learning: A Review,'' International Journal of Informatics, Information System and Computer Engineering (INJIISCOM), vol. 5, no. 1, pp. 53–87, Mar. 2024. DOI: https://doi.org/10.34010/injiiscom.v5i1.12145
E. S. Agung, A. P. Rifai, and T. Wijayanto, ''Image-based facial emotion recognition using convolutional neural network on emognition dataset,'' Scientific Reports, vol. 14, no. 1, June 2024, Art. no. 14429. DOI: https://doi.org/10.1038/s41598-024-65276-x
M. Talele, R. Jain, and P. Kulkarni, ''Review of Face Emotion Recognition Using Feature Extraction Techniques,'' in 2023 Intelligent Computing and Control for Engineering and Business Systems (ICCEBS), Chennai, India, Sept. 2023, pp. 1–6. DOI: https://doi.org/10.1109/ICCEBS58601.2023.10448632
H. R. Conte and R. Plutchik, ''A circumplex model for interpersonal personality traits.,'' Journal of Personality and Social Psychology, vol. 40, no. 4, pp. 701–711, Apr. 1981. DOI: https://doi.org/10.1037//0022-3514.40.4.701
M. Talele and R. Jain, ''A Comparative Analysis of CNNs and ResNet50 for Facial Emotion Recognition,'' Engineering, Technology & Applied Science Research, vol. 15, no. 2, pp. 20693–20701, Apr. 2025. DOI: https://doi.org/10.48084/etasr.9849
M. Talele and R. Jain, ''Complex Face Emotion Recognition Using Convolutional Neural Networks,'' Journal Européen des Systèmes Automatisés, vol. 58, no. 1, pp. 171–178, Jan. 2025. DOI: https://doi.org/10.18280/jesa.580119
M. Mascaró-Oliver, E. Amengual-Alcover, M. F. Roig-Maimó, and R. Mas-Sansó, ''UIBVFEDPlus-Light: Virtual facial expression dataset with lighting,'' PLOS ONE, vol. 18, no. 9, Sept. 2023, Art. no. e0287006. DOI: https://doi.org/10.1371/journal.pone.0287006
C. C. F. Or, D. Y. Lim, S. Chen, and A. L. F. Lee, ''Face Recognition Under Adverse Viewing Conditions: Implications for Eyewitness Testimony,'' Policy Insights from the Behavioral and Brain Sciences, vol. 10, no. 2, pp. 264–271, Oct. 2023. DOI: https://doi.org/10.1177/23727322231194458
S. Saxena, S. Tripathi, and T. S. B. Sudarshan, ''Deep Dive into Faces: Pose & Illumination Invariant Multi-Face Emotion Recognition System,'' in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, Nov. 2019, pp. 1088–1093. DOI: https://doi.org/10.1109/IROS40897.2019.8967874
X. Xie et al., ''Affective Impressions Recognition under Different Colored Lights Based on Physiological Signals and Subjective Evaluation Method,'' Sensors, vol. 23, no. 11, June 2023, Art. no. 5322. DOI: https://doi.org/10.3390/s23115322
A. Bonassi et al., ''The Recognition of Cross-Cultural Emotional Faces Is Affected by Intensity and Ethnicity in a Japanese Sample,'' Behavioral Sciences, vol. 11, no. 5, Apr. 2021, Art. no. 59. DOI: https://doi.org/10.3390/bs11050059
D. Schneevogt and P. Paggio, ''The Effect of Gender and Age Differences on the Recognition of Emotions from Facial Expressions,'' in Proceedings of the Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media (PEOPLES), Osaka, Japan, Sept. 2016, pp. 11–19.
X. Wang, J. Jia, P. Hu, S. Wu, J. Tang, and L. Cai, ''Understanding the emotional impact of images,'' in Proceedings of the 20th ACM International Conference on Multimedia, Nara, Japan, Oct. 2012, pp. 1369–1370. DOI: https://doi.org/10.1145/2393347.2396489
''Face expression recognition dataset.'' Kaggle, Available: https://www.kaggle.com/datasets/jonathanoheix/face-expression-recognition-dataset.
M. S. Benda and K. S. Scherf, ''The Complex Emotion Expression Database: A validated stimulus set of trained actors,'' PLOS ONE, vol. 15, no. 2, Feb. 2020, Art. no. e0228248. DOI: https://doi.org/10.1371/journal.pone.0228248
M. Mujiyanto, A. Setyanto, K. Kusrini, and E. Utami, ''Swin Transformer with Enhanced Dropout and Layer-wise Unfreezing for Facial Expression Recognition in Mental Health Detection,'' Engineering, Technology & Applied Science Research, vol. 14, no. 6, pp. 19016–19023, Dec. 2024. DOI: https://doi.org/10.48084/etasr.9139
Downloads
How to Cite
License
Copyright (c) 2026 Rajashree Jain, Milind Talele, Taher Muhammad Ali
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
