Development of the ColorEmoNet Dataset and of a Machine Learning Approach for Emotion Recognition Using Color Theory
Received: 8 May 2025 | Revised: 1 July 2025, 31 July 2025, and 11 August 2025 | Accepted: 14 August 2025 | Online: 8 December 2025
Corresponding author: Mayuri Manoj Bapat
Abstract
Color theory is a way of understanding how colors influence human perception and emotions. Color theory and Machine Learning (ML) can be combined to deeply understand the psychological impact of colors on human emotion. This study aims to expand our understanding of the color-emotion relationships. To establish a relationship, ML algorithms, such as Support Vector Machines (SVM), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN), were employed to classify the emotional feedback induced by colors in images using the ColorEmoNet dataset. The proposed approach achieved an accuracy of 76% using SVM, 86.49% using CNN, and 68% using RNN, demonstrating the effectiveness of color-based features for emotion recognition. The results of this study can contribute to developing innovative emotion recognition systems that enhance emotional understanding in practical applications.
Keywords:
ColorEmoNet, color theory, emotion recognition, machine learning, deep learningDownloads
References
R. W. Levenson, J. Soto, and N. Pole, "Emotion, biology, and culture," in Handbook of cultural psychology, New York, NY, USA: The Guilford Press, 2007, pp. 780–796.
J. Palsa, J. Hurtuk, E. Chovancova, and L. Vaniscak, "Emotion Detection as a Supportive Tool in Color Therapy," in 2021 19th International Conference on Emerging eLearning Technologies and Applications (ICETA), Košice, Slovakia, Nov. 2021, pp. 287–292. DOI: https://doi.org/10.1109/ICETA54173.2021.9726576
A. Kaur, "A Link Between Colors and Emotions; A Study of Undergraduate Females," International Journal of Engineering Research and Technology, vol. 9, no. 09, pp. 553–557, Sep. 2020. DOI: https://doi.org/10.17577/IJERTV9IS090319
A. Sartori, D. Culibrk, Y. Yan, and N. Sebe, "Who’s Afraid of Itten: Using the Art Theory of Color Combination to Analyze Emotions in Abstract Paintings," in Proceedings of the 23rd ACM International Conference on Multimedia, Jul. 2015, pp. 311–320. DOI: https://doi.org/10.1145/2733373.2806250
M. Rai, T. Maity, R. K. Yadav, and S. Yadav, "A Review on Detection of Human Emotions Using Colored and Infrared Images." Social Science Research Network, Jul. 14, 2022. DOI: https://doi.org/10.2139/ssrn.4161798
M. Eyza, "Facial Emotion Detection for Educational Purpose Using Image Processing Technique," B.S. Thesis, Universiti Teknologi Mara, Malaysia, 2020.
G. Vijayanand, B. Hari, S. Karthick, and V. Jaikrishnan, "Emotion Detection using Machine Learning," International Journal of Engineering Research, vol. 8, no. 8, 2020.
N. Mehendale, "Facial emotion recognition using convolutional neural networks (FERC)," SN Applied Sciences, vol. 2, no. 3, Mar. 2020, Art. no. 446. DOI: https://doi.org/10.1007/s42452-020-2234-1
F. Julin, "Vision based facial emotion detection using deep convolutional neural networks," B.S. Thesis, Mälardalen University, Sweden, 2019.
S. Gupta and S. K. Gupta, "Investigating Emotion-Color Association in Deep Neural Networks." arXiv, Nov. 22, 2020.
D. L. Spiers, "Facial emotion detection using deep learning," B.S. Thesis, Uppsala University, Sweden, 2016.
H. I. Dino and M. B. Abdulrazzaq, "Facial Expression Classification Based on SVM, KNN and MLP Classifiers," in 2019 International Conference on Advanced Science and Engineering (ICOASE), Zakho - Duhok, Iraq, Apr. 2019, pp. 70–75. DOI: https://doi.org/10.1109/ICOASE.2019.8723728
S. N. Shivhare and S. K. Saritha, "Emotion Detection From Text Documents," International Journal of Data Mining & Knowledge Management Process, vol. 4, no. 6, pp. 51–57, Nov. 2014. DOI: https://doi.org/10.5121/ijdkp.2014.4605
A. Upadhyay, A. Tyagi, M. Tyagi, N. Dhiman, and M. K. Sharma, "Color and Psychological Functioning with LSTM: The Impact of Colors on Emotional Quotient," Journal of Positive School Psychology, vol. 6, no. 6, pp. 4773–4778, 2022.
A. A. Mande, S. Dani, S. Telang, and Z. Shao, "Emotion Detection Using Audio Data Samples," International Journal of Advanced Research in Computer Science, vol. 10, no. 6, pp. 13–20, Dec. 2019. DOI: https://doi.org/10.26483/ijarcs.v10i6.6489
B. Ranjgar, M. Khoshlahjeh Azar, A. Sadeghi-Niaraki, and S. M. Choi, "A Novel Method for Emotion Extraction From Paintings Based on Luscher’s Psychological Color Test: Case Study Iranian-Islamic Paintings," IEEE Access, vol. 7, pp. 120857–120871, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2936896
S. Ryoo, "Emotion Affective Color Transfer," International Journal of Software Engineering and Its Applications, vol. 8, no. 3, pp. 227–232, Mar. 2014.
C. S. Son and H. M. Chung, "An Emotion Classification Based on Fuzzy Inference and Color Psychology," International Journal of Fuzzy Logic and Intelligent Systems, vol. 4, no. 1, pp. 18–22, Jun. 2004. DOI: https://doi.org/10.5391/IJFIS.2004.4.1.018
M. Muratbekova and P. Shamoi, "Color-Emotion Associations in Art: Fuzzy Approach," IEEE Access, vol. 12, pp. 37937–37956, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3375361
C. F. Hibadullah, A. W. C. Liew, and J. Jo, "Colour-emotion association study on abstract art painting," in 2015 International Conference on Machine Learning and Cybernetics (ICMLC), Guangzhou, China, Jul. 2015, pp. 488–493. DOI: https://doi.org/10.1109/ICMLC.2015.7340605
S. Liu and M. Pei, "Texture-Aware Emotional Color Transfer Between Images," IEEE Access, vol. 6, pp. 31375–31386, 2018. DOI: https://doi.org/10.1109/ACCESS.2018.2844540
S. Carvalho, J. Leite, S. Galdo-Álvarez, and Ó. F. Gonçalves, "The Emotional Movie Database (EMDB): A Self-Report and Psychophysiological Study," Applied Psychophysiology and Biofeedback, vol. 37, no. 4, pp. 279–294, Dec. 2012. DOI: https://doi.org/10.1007/s10484-012-9201-6
T. Lee, N. Lee, S. Seo, and D. Kang, "A Study on the Prediction of Emotion from Image by Time-flow depend on Color Analysis," in 2020 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, Dec. 2020, pp. 747–749. DOI: https://doi.org/10.1109/CSCI51800.2020.00141
J. Wei, X. Yang, and Y. Dong, "User-generated video emotion recognition based on key frames," Multimedia Tools and Applications, vol. 80, no. 9, pp. 14343–14361, Apr. 2021. DOI: https://doi.org/10.1007/s11042-020-10203-1
S. Tripathi, S. Acharya, R. Sharma, S. Mittal, and S. Bhattacharya, "Using Deep and Convolutional Neural Networks for Accurate Emotion Classification on DEAP Data," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 2, pp. 4746–4752, Feb. 2017. DOI: https://doi.org/10.1609/aaai.v31i2.19105
S. K. Khare, A. Nishad, A. Upadhyay, and V. Bajaj, "Classification of emotions from EEG signals using time-order representation based on the S-transform and convolutional neural network," Electronics Letters, vol. 56, no. 25, pp. 1359–1361, 2020. DOI: https://doi.org/10.1049/el.2020.2380
N. Yousefi, M. C. Cakmak, and N. Agarwal, "Examining Multimodel Emotion Assessment and Resonance with Audience on YouTube," in Proceedings of the 2024 9th International Conference on Multimedia and Image Processing, May 2024, pp. 85–93. DOI: https://doi.org/10.1145/3665026.3665039
M. Li et al., "A decision support system using hybrid AI based on multi-image quality model and its application in color design," Future Generation Computer Systems, vol. 113, pp. 70–77, Dec. 2020. DOI: https://doi.org/10.1016/j.future.2020.06.034
R. Kishore Kanna, B. S. Panigrahi, S. K. Sahoo, A. R. Reddy, Y. Manchala, and N. K. Swain, "CNN Based Face Emotion Recognition System for Healthcare Application," EAI Endorsed Transactions on Pervasive Health and Technology, vol. 10, Mar. 2024. DOI: https://doi.org/10.4108/eetpht.10.5458
M. C. Cakmak, M. Shaik, and N. Agarwal, "Emotion Assessment of YouTube Videos using Color Theory," in Proceedings of the 2024 9th International Conference on Multimedia and Image Processing, May 2024, pp. 6–14. DOI: https://doi.org/10.1145/3665026.3665028
A. Wędołowska, D. Weber, and B. Kostek, "Predicting Emotion From Color Present in Images and Video Excerpts by Machine Learning," IEEE Access, vol. 11, pp. 66357–66373, 2023. DOI: https://doi.org/10.1109/ACCESS.2023.3289713
V. Bekhtereva and M. M. Müller, "Bringing color to emotion: The influence of color on attentional bias to briefly presented emotional images," Cognitive, Affective, & Behavioral Neuroscience, vol. 17, no. 5, pp. 1028–1047, Oct. 2017. DOI: https://doi.org/10.3758/s13415-017-0530-z
S. Chopparapu and J. B. Seventline, "An Efficient Multi-modal Facial Gesture-based Ensemble Classification and Reaction to Sound Framework for Large Video Sequences," Engineering, Technology & Applied Science Research, vol. 13, no. 4, pp. 11263–11270, Aug. 2023. DOI: https://doi.org/10.48084/etasr.6087
P. Kulkarni and T. M. Rajesh, "Analysis on Techniques Used to Recognize and Identifying the Human Emotions," International Journal of Electrical and Computer Engineering, vol. 10, no. 3, pp. 3307–3314, 2020. DOI: https://doi.org/10.11591/ijece.v10i3.pp3307-3314
A. Saxena, A. Khanna, and D. Gupta, "Emotion Recognition and Detection Methods: A Comprehensive Survey," Journal of Artificial Intelligence and Systems, vol. 2, no. 1, pp. 53–79, Feb. 2020. DOI: https://doi.org/10.33969/AIS.2020.21005
G. Chhabra, E. M. Onyema, S. Kumar, M. Goutham, S. Mandapati, and C. Iwendi, "Human Emotions Recognition, Analysis and Transformation by the Bioenergy Field in Smart Grid Using Image Processing," Electronics, vol. 11, no. 23, Jan. 2022, Art. no. 4059. DOI: https://doi.org/10.3390/electronics11234059
A. S. Imran, S. M. Daudpota, Z. Kastrati, and R. Batra, "Cross-Cultural Polarity and Emotion Detection Using Sentiment Analysis and Deep Learning on COVID-19 Related Tweets," IEEE Access, vol. 8, pp. 181074–181090, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.3027350
W. L. Zheng, W. Liu, Y. Lu, B. L. Lu, and A. Cichocki, "EmotionMeter: A Multimodal Framework for Recognizing Human Emotions," IEEE Transactions on Cybernetics, vol. 49, no. 3, pp. 1110–1122, Mar. 2019. DOI: https://doi.org/10.1109/TCYB.2018.2797176
E. Reynolds, "The communication of emotion across languages and cultures: an exploration of display rules in foreign language learning," Ph.D. dissertation, University of Illinois at Urbana-Champaign, IL, USA, 2012.
A. MirMashhouri, A. Bastanfard, and D. Amirkhani, "Collecting a database for emotional responses to simple and patterned two-color images," Multimedia Tools and Applications, vol. 81, no. 13, pp. 18935–18953, May 2022. DOI: https://doi.org/10.1007/s11042-022-11966-5
I. A. Surov, "Quantum Core Affect. Color-Emotion Structure of Semantic Atom," Frontiers in Psychology, vol. 13, 2022, Art. no. 838029. DOI: https://doi.org/10.3389/fpsyg.2022.838029
S. Yang, L. Wenhui, W. Jie, and Y. Xuezhi, "Combining MRF and ν-SVM for SAR sea ice image classification," National Remote Sensing Bulletin, vol. 19, no. 5, pp. 844–855, 2015. DOI: https://doi.org/10.11834/jrs.20154206
D. Jonauskaite et al., "A machine learning approach to quantify the specificity of colour–emotion associations and their cultural differences," Royal Society Open Science, vol. 6, no. 9, Sep. 2019, Art. no. 190741. DOI: https://doi.org/10.1098/rsos.190741
S. Mali and M. B. Mayuri, "ColorEmoNet." Mendeley Data, Jun. 26, 2025.
A. Takei and S. Imaizumi, "Effects of color–emotion association on facial expression judgments," Heliyon, vol. 8, no. 1, Jan. 2022. DOI: https://doi.org/10.1016/j.heliyon.2022.e08804
N. Kaya and H. H. Epps, "Relationship between color and emotion: A study of college students," College Student Journal, vol. 38, no. 3, pp. 396–405, 2004.
M. Hanada, "Correspondence analysis of color–emotion associations," Color Research & Application, vol. 43, no. 2, pp. 224–237, 2018. DOI: https://doi.org/10.1002/col.22171
A. Sherstinsky, "Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network," Physica D: Nonlinear Phenomena, vol. 404, Mar. 2020, Art. no. 132306. DOI: https://doi.org/10.1016/j.physd.2019.132306
M. V. Sokolova, A. Fernández-Caballero, L. Ros, J. M. Latorre, and J. P. Serrano, "Evaluation of Color Preference for Emotion Regulation," in Artificial Computation in Biology and Medicine, vol. 9107, J. M. Ferrández Vicente, J. R. Álvarez-Sánchez, F. De La Paz López, Fco. J. Toledo-Moreo, and H. Adeli, Eds. Springer International Publishing, 2015, pp. 479–487. DOI: https://doi.org/10.1007/978-3-319-18914-7_50
Downloads
How to Cite
License
Copyright (c) 2025 Mayuri Manoj Bapat, S. M. Mali, C. H. Patil
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
