Enhancing Arabic Fake News Detection: Evaluating Data Balancing Techniques Across Multiple Machine Learning Models
Received: 3 June 2024 | Revised: 20 June 2024 | Accepted: 27 June 2024 | Online: 8 July 2024
Corresponding author: Eman Aljohani
Abstract
The spread of fake news has become a serious concern in the era of rapid information dissemination through social networks, especially when it comes to Arabic-language content, where automated detection systems are not as advanced as those for English-language content. This study evaluates the effectiveness of various data balancing techniques, such as class weights, random under-sampling, SMOTE, and SMOTEENN, across multiple machine learning models, namely XGBoost, Random Forest, CNN, BIGRU, BILSTM, CNN-LSTM, and CNN-BIGRU, to address the critical challenge of dataset imbalance in Arabic fake news detection. Accuracy, AUC, precision, recall, and F1-score were used to evaluate the performance of these models on balanced and imbalanced datasets. The results show that SMOTEENN greatly improves model performance, especially the F1-score, precision, and recall. In addition to advancing the larger objective of preserving information credibility on social networks, this study emphasizes the need for advanced data balancing strategies to improve Arabic fake news detection systems.
Keywords:
text classification, fake news, imbalanced datasets, balancing techniquesDownloads
References
L. Alsudias and P. Rayson, "COVID-19 and Arabic Twitter: How can Arab World Governments and Public Health Organizations Learn from Social Media?," in Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020, Online, Apr. 2020, [Online]. Available: https://aclanthology.org/2020.nlpcovid19-acl.16.
B. Ahmed, G. Ali, A. Hussain, A. Baseer, and J. Ahmed, "Analysis of Text Feature Extractors using Deep Learning on Fake News," Engineering, Technology & Applied Science Research, vol. 11, no. 2, pp. 7001–7005, Apr. 2021.
S. Alqurashi, B. Hamoui, A. Alashaikh, A. Alhindi, and E. Alanazi, "Eating Garlic Prevents COVID-19 Infection: Detecting Misinformation on the Arabic Content of Twitter," arXiv.org, Jan. 09, 2021. https://arxiv.org/abs/2101.05626v1.
K. M. Fouad, S. F. Sabbeh, and W. Medhat, "Arabic fake news detection using deep learning," Computers, Materials and Continua, vol. 71, no. 2, pp. 3647–3665, 2022.
S. Alyoubi, M. Kalkatawi, and F. Abukhodair, "The Detection of Fake News in Arabic Tweets Using Deep Learning," Applied Sciences, vol. 13, no. 14, Jan. 2023, Art. no. 8209.
S. Bhattacharjee, S. Maity, and S. Chatterjee, "Addressing Class Imbalance in Fake News Detection with Latent Space Resampling," in Computational Intelligence in Pattern Recognition, Kolkata, India, 2023, pp. 427–438.
A. Khalil, M. Jarrah, and M. Aldwairi, "Hybrid Neural Network Models for Detecting Fake News Articles," Human-Centric Intelligent Systems, vol. 4, no. 1, pp. 136–146, Mar. 2024.
B. S. Arkok and A. M. Zeki, "Classification of Quranic topics based on imbalanced classification," Indonesian Journal of Electrical Engineering and Computer Science, vol. 22, no. 2, pp. 678–687, May 2021.
S. Al-Azani and E. S. M. El-Alfy, "Using Word Embedding and Ensemble Learning for Highly Imbalanced Data Sentiment Analysis in Short Arabic Text," Procedia Computer Science, vol. 109, pp. 359–366, Jan. 2017.
P. Jeatrakul, K. W. Wong, and C. C. Fung, "Classification of Imbalanced Data by Combining the Complementary Neural Network and SMOTE Algorithm," in Neural Information Processing. Models and Applications, Sydney, Australia, 2010, pp. 152–159.
A. B. Nassif, A. Elnagar, O. Elgendy, and Y. Afadar, "Arabic fake news detection based on deep contextualized embedding models," Neural Computing and Applications, vol. 34, no. 18, pp. 16019–16032, Sep. 2022.
F. Haouari, M. Hasanain, R. Suwaileh, and T. Elsayed, "ArCOV-19: The First Arabic COVID-19 Twitter Dataset with Propagation Networks," in Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Virtual), Dec. 2021, pp. 82–91. [Online]. Available: https://aclanthology.org/2021.wanlp-1.9.
F. Haouari, M. Hasanain, R. Suwaileh, and T. Elsayed, "ArCOV19-Rumors: Arabic COVID-19 Twitter Dataset for Misinformation Detection," in Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Virtual), Dec. 2021, pp. 72–81. [Online]. Available: https://aclanthology.org/2021.wanlp-1.8.
M. S. Hadj Ameur and H. Aliane, "AraCOVID19-MFH: Arabic COVID-19 Multi-label Fake News & Hate Speech Detection Dataset," Procedia Computer Science, vol. 189, pp. 232–241, Jan. 2021.
F. Alam et al., "Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society," in Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, Dominican Republic, Aug. 2021, pp. 611–649.
H. Mubarak and S. Hassan, "ArCorona: Analyzing Arabic Tweets in the Early Days of Coronavirus (COVID-19) Pandemic," in Proceedings of the 12th International Workshop on Health Text Mining and Information Analysis, online, Dec. 2021. [Online]. Available: https://aclanthology.org/2021.louhi-1.1.
A. Khalil, M. Jarrah, M. Aldwairi, and M. Jaradat, "AFND: Arabic fake news dataset for the detection and classification of articles credibility," Data in Brief, vol. 42, Jun. 2022, Art. no. 108141.
A. Khalil, M. Jarrah, M. Aldwairi, and Y. Jararweh, "Detecting Arabic Fake News Using Machine Learning," in 2021 Second International Conference on Intelligent Data Science Technologies and Applications (IDSTA), Tartu, Estonia, Nov. 2021, pp. 171–177.
A. R. Mahlous and A. Al-Laith, "Fake News Detection in Arabic Tweets during the COVID-19 Pandemic," International Journal of Advanced Computer Science and Applications, vol. 12, no. 6, 2021.
F. M. U. Baran, L. S. A. Alzughaybi, M. A. S. Bajafar, M. N. M. Alsaedi, T. F. H. Serdar, and O. M. N. Mirza, "Etiqa’a: An Android Mobile Application for Monitoring Teen’s Private Messages on WhatsApp to Detect Harmful/Inappropriate Words in Arabic using Machine Learning," Engineering, Technology & Applied Science Research, vol. 13, no. 6, pp. 12012–12019, Dec. 2023.
T. Feizi, M. H. Moattar, and H. Tabatabaee, "A multi-manifold learning based instance weighting and under-sampling for imbalanced data classification problems," Journal of Big Data, vol. 10, no. 1, Oct. 2023, Art. no. 153.
M. Azadbakht, C. S. Fraser, and K. Khoshelham, "Synergy of sampling techniques and ensemble classifiers for classification of urban environments using full-waveform LiDAR data," International Journal of Applied Earth Observation and Geoinformation, vol. 73, pp. 277–291, Dec. 2018.
T. Hasanin, T. M. Khoshgoftaar, J. L. Leevy, and R. A. Bauder, "Severely imbalanced Big Data challenges: investigating data sampling approaches," Journal of Big Data, vol. 6, no. 1, Nov. 2019, Art. no. 107.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "SMOTE: Synthetic Minority Over-sampling Technique," Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, Jun. 2002.
G. E. A. P. A. Batista, R. C. Prati, and M. C. Monard, "A study of the behavior of several methods for balancing machine learning training data," SIGKDD Explorations Newsletter, vol. 6, no. 1, pp. 20–29, Mar. 2004.
Downloads
How to Cite
License
Copyright (c) 2024 Eman Aljohani
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
