Assessing Institutional Performance using Machine Learning on Arabic Facebook Comments
Received: 10 June 2024 | Revised: 27 June 2024 | Accepted: 30 June 2024 | Online: 2 August 2024
Corresponding author: Zainab Alwan Anwer
Abstract
Social networks have become increasingly influential in shaping political and governmental decisions in Middle Eastern countries and worldwide. Facebook is considered one of the most popular social media platforms in Iraq. Exploiting such a platform to assess the performance of institutions remains underutilized. This study proposes a model to help institutions, such as the Iraqi Ministry of Justice, evaluate their performance based on sentiment analysis on Facebook. Different machine learning algorithms were used, such as Support Vector Machine (SVM), Logistic Regression (LR), Extreme Gradient Boosting (XGBoost), Naive Bayes (NB), and Random Forest (RF). Extensive experimental analysis was performed using a large dataset extracted from Facebook pages belonging to the Iraqi Ministry of Justice. The results showed that SVM achieved the highest accuracy of 97.774% after retaining certain stop words, which proved to have a significant impact on the accuracy of the algorithms, ensuring the correct classification of comments while preserving the sentence's meaning.
Keywords:
Facebook, TF-IDF, sentiment analysis, social media, machine learningDownloads
References
I. M. Tarigan, M. A. K. Harahap, D. M. Sari, R. D. Sakinah, and A. M. A. Ausat, "Understanding Social Media: Benefits of Social Media for Individuals," Jurnal Pendidikan Tambusai, vol. 7, no. 1, pp. 2317–2322, Feb. 2023.
A. Yohanna, "The influence of social media on social interactions among students," Indonesian Journal of Social Sciences, vol. 12, no. 2, pp. 34–48, 2020.
G. Appel, L. Grewal, R. Hadi, and A. T. Stephen, "The future of social media in marketing," Journal of the Academy of Marketing Science, vol. 48, no. 1, pp. 79–95, Jan. 2020.
Noureen, S. H. H. Huspi, and Z. Ali, "Sentiment Analysis on Roman Urdu Students' Feedback Using Enhanced Word Embedding Technique," Baghdad Science Journal, vol. 21, no. 2(SI), Feb. 2024.
K. F. Ferine, S. S. Gadzali, A. M. A. Ausat, M. Marleni, and D. M. Sari, "The Impact of Social Media on Consumer Behavior," Community Development Journal : Jurnal Pengabdian Masyarakat, vol. 4, no. 1, pp. 843–847, Mar. 2023.
A. M. A. Ausat, "The Role of Social Media in Shaping Public Opinion and Its Influence on Economic Decisions," Technology and Society Perspectives (TACIT), vol. 1, no. 1, pp. 35–44, Aug. 2023.
"Top Social Media Networks Websites Ranking in Iraq in May 2024," Similarweb. https://www.similarweb.com/top-websites/iraq/computers-electronics-and-technology/social-networks-and-online-communities/.
L. A. Habeeb, "Sentiment Analysis for Iraqis Dialect in Social Media," Iraqi Journal of Information and Communication Technology, vol. 1, no. 2, pp. 24–32, Jul. 2018.
A. H. Ombabi, W. Ouarda, and A. M. Alimi, "Deep learning CNN–LSTM framework for Arabic sentiment analysis using textual information shared in social networks," Social Network Analysis and Mining, vol. 10, no. 1, Jul. 2020, Art. no. 53.
H. AlSalman, "An Improved Approach for Sentiment Analysis of Arabic Tweets in Twitter Social Media," in 2020 3rd International Conference on Computer Applications & Information Security (ICCAIS), Riyadh, Saudi Arabia, Mar. 2020.
N. K. Bolbol and A. Y. Maghari, "Sentiment Analysis of Arabic Tweets Using Supervised Machine Learning," in 2020 International Conference on Promising Electronic Technologies (ICPET), Jerusalem, Palestine, Dec. 2020, pp. 89–93.
M. Alzyout, E. Al Bashabsheh, H. Najadat, and A. Alaiad, "Sentiment Analysis of Arabic Tweets about Violence Against Women using Machine Learning," in 2021 12th International Conference on Information and Communication Systems (ICICS), Valencia, Spain, May 2021, pp. 171–176.
F. Alderazi, A. A. Algosaibi, and M. A. Alabdullatif, "The Use of Arabic Language COVID-19 Tweets Analysis in IoT Applications," in 2021 IEEE Global Conference on Artificial Intelligence and Internet of Things (GCAIoT), Dubai, United Arab Emirates, Dec. 2021, pp. 112–117.
R. Kharsa and S. Harous, "Machine Learning Classification Algorithms for Sentiment Analysis in Arabic: Performance Evaluation and Comparison," in 2022 International Conference on Electrical and Computing Technologies and Applications (ICECTA), Ras Al Khaimah, United Arab Emirates, Nov. 2022, pp. 395–400.
A. Alqarni and A. Rahman, "Arabic Tweets-Based Sentiment Analysis to Investigate the Impact of COVID-19 in KSA: A Deep Learning Approach," Big Data and Cognitive Computing, vol. 7, no. 1, Mar. 2023, Art. no. 16.
M. Faisal, Z. Abouelhassan, F. Alotaibi, R. Alsaeedi, F. Alazmi, and S. Alkanadari, "Sentiment Analysis Using Machine Learning Model for Qatar World Cup 2022 among Different Arabic Countries Using Twitter API," in 2023 IEEE World AI IoT Congress (AIIoT), Seattle, WA, USA, Jun. 2023, pp. 222–228.
A. S. Abdalrada, J. Abawajy, T. Al-Quraishi, and S. M. S. Islam, "Prediction of cardiac autonomic neuropathy using a machine learning model in patients with diabetes," Therapeutic Advances in Endocrinology and Metabolism, vol. 13, Jan. 2022, Art. no. 20420188221086693.
A. S. Abdalrada, J. H. Abawajy, M. U. Chowdhury, S. Rajasegarar, T. Al-Quraishi, and H. F. Jelinek, "Relationship Between Angiotensin Converting Enzyme Gene and Cardiac Autonomic Neuropathy Among Australian Population," in Recent Advances on Soft Computing and Data Mining, 2018, pp. 135–146.
A. S. Abdalrada, J. Abawajy, M. Chowdhury, S. Rajasegarar, T. Al-Quraishi, and H. F. Jelinek, "Meta learning ensemble technique for diagnosis of cardiac autonomic neuropathy based on heart rate variability features," in 30th International Conference on Computer Applications in Industry and Engineering, CAINE 2017, 2017, pp. 169–175.
T. Al-Quraishi, J. H. Abawajy, N. Al-Quraishi, A. Abdalrada, and L. Al-Omairi, "Predicting Breast Cancer Risk Using Subset of Genes," in 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT), Paris, France, Apr. 2019, pp. 1379–1384.
A. Abdalrada, Ali Fahem Neamah, and Hayder Murad, "Predicting Diabetes Disease Occurrence Using Logistic Regression: An Early Detection Approach," Iraqi Journal For Computer Science and Mathematics, vol. 5, no. 1, pp. 160–167, Jan. 2024.
P. Karthika, R. Murugeswari, and R. Manoranjithem, "Sentiment Analysis of Social Media Network Using Random Forest Algorithm," in 2019 IEEE International Conference on Intelligent Techniques in Control, Optimization and Signal Processing (INCOS), Tamilnadu, India, Apr. 2019.
H. A. Alatabi and A. R. Abbas, "Sentiment Analysis in Social Media using Machine Learning Techniques," Iraqi Journal of Science, pp. 193–201, Jan. 2020.
H. Alamoudi et al., "Arabic Sentiment Analysis for Student Evaluation using Machine Learning and the AraBERT Transformer," Engineering, Technology & Applied Science Research, vol. 13, no. 5, pp. 11945–11952, Oct. 2023.
D. Elangovan and V. Subedha, "Adaptive Particle Grey Wolf Optimizer with Deep Learning-based Sentiment Analysis on Online Product Reviews," Engineering, Technology & Applied Science Research, vol. 13, no. 3, pp. 10989–10993, Jun. 2023.
M. Nandan, S. Chatterjee, A. Parai, and O. Bagchi, "Sentiment Analysis of Twitter Classification by Applying Hybrid-Based Techniques," in Proceedings of the 3rd International Conference on Communication, Devices and Computing, 2022, pp. 591–606.
M. A. Kausar, S. O. Fageeri, and A. Soosaimanickam, "Sentiment Classification based on Machine Learning Approaches in Amazon Product Reviews," Engineering, Technology & Applied Science Research, vol. 13, no. 3, pp. 10849–10855, Jun. 2023.
E. Refaee, "Sentiment Analysis for Micro-blogging Platforms in Arabic," in Social Computing and Social Media. Applications and Analytics, Vancouver, Canada, 2017, pp. 275–294.
Z. Lu, "Web Page Classification Using Features from Titles and Snippets," M.S. Thesis, University of Ottawa, 2015.
J. Ramos, "Using tf-idf to determine word relevance in document queries," in Proceedings of the first instructional conference on machine learning, 2003, vol. 242, no. 1, pp. 29–48.
S. A. Aljuhani and N. Alghamdi, "A comparison of sentiment analysis methods on Amazon reviews of Mobile Phones," International Journal of Advanced Computer Science and Applications, vol. 10, no. 6, pp. 608–617, 2019.
H. Syahputra and A. Wibowo, "Comparison of Support Vector Machine (SVM) and Random Forest Algorithm for Detection of Negative Content on Websites," Jurnal Ilmiah Teknik Elektro Komputer dan Informatika, vol. 9, no. 1, pp. 165–173, 2023.
B. E. Boser, I. M. Guyon, and V. N. Vapnik, "A training algorithm for optimal margin classifiers," in Proceedings of the fifth annual workshop on Computational learning theory, Pittsburgh, PA, USA, Apr. 1992, pp. 144–152.
C. Cortes and V. Vapnik, "Support-vector networks," Machine Learning, vol. 20, no. 3, pp. 273–297, Sep. 1995.
A. Goel, J. Gautam, and S. Kumar, "Real time sentiment analysis of tweets using Naive Bayes," in 2016 2nd International Conference on Next Generation Computing Technologies (NGCT), Dehradun, India, Oct. 2016, pp. 257–261.
A. A. Farisi, Y. Sibaroni, and S. A. Faraby, "Sentiment analysis on hotel reviews using Multinomial Naïve Bayes classifier," Journal of Physics: Conference Series, vol. 1192, no. 1, Nov. 2019, Art. no. 012024.
K. Dhola and M. Saradva, "A Comparative Evaluation of Traditional Machine Learning and Deep Learning Classification Techniques for Sentiment Analysis," in 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Noida, India, Jan. 2021, pp. 932–936.
T. Chen and C. Guestrin, "XGBoost: A Scalable Tree Boosting System," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, May 2016, pp. 785–794.
B. Gaye and A. Wulamu, "Sentimental Analysis for Online Reviews using Machine Learning Algorithms," International Research Journal of Engineering and Technology, vol. 6, no. 8, pp. 1270–1275, 2019.
L. Breiman, "Random Forests," Machine Learning, vol. 45, no. 1, pp. 5–32, Oct. 2001.
R. S. Utsha, M. Keya, A. Hasan, and S. Islam, "Qword at CheckThat! 2021: An Extreme Gradient Boosting Approach for Multiclass Fake News Detection," in CEUR Workshop Proceedings, Bucharest, Romania.
A. Poornima and K. S. Priya, "A Comparative Sentiment Analysis Of Sentence Embedding Using Machine Learning Techniques," in 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, Mar. 2020, pp. 493–496.
"Export Facebook, Instagram, Twitter, YouTube, TikTok, Vimeo Comments," exportcomments.com. https://exportcomments.com/.
S. Dutt, S. Chandramouli, and A. K. Das, Machine Learning, 1st ed. Pearson Education, 2018.
H. Smolic, "The Importance of Cross-Validation in Machine Learning," Feb. 19, 2024. https://graphite-note.com/the-importance-of-cross-validation-in-machine-learning/.
A. Tripathy, A. Agrawal, and S. K. Rath, "Classification of Sentimental Reviews Using Machine Learning Techniques," Procedia Computer Science, vol. 57, pp. 821–829, Jan. 2015.
A. S. Al-Jumaili, "A Hybrid Method of Linguistic and Statistical Features for Arabic Sentiment Analysis," Baghdad Science Journal, vol. 17, no. 1(Suppl.), pp. 385–390, Mar. 2020.
Downloads
How to Cite
License
Copyright (c) 2024 Zainab Alwan Anwer, Ahmad Shaker Abdalrada
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
