Enhancing Fake News Detection with Transformer Models and Summarization

Authors

  • Abdelhalim Saadi Faculty of Technology, Setif 1 University – Ferhat Abbas, Algeria
  • Hacene Belhadef Department of Fundamental Computing and its Applications, Faculty of New Technologies of Information and Communication, University of Abdelhamid Mehri – Constantine 2, Algeria
  • Akram Guessas Department of Fundamental Computing and its Applications, Faculty of New Technologies of Information and Communication, University of Abdelhamid Mehri – Constantine 2, Algeria
  • Oussama Hafirassou Department of Fundamental Computing and its Applications, Faculty of New Technologies of Information and Communication, University of Abdelhamid Mehri – Constantine 2, Algeria
Volume: 15 | Issue: 3 | Pages: 23253-23259 | June 2025 | https://doi.org/10.48084/etasr.10678

Received: 22 February 2025 | Revised: 23 March 2025 and 14 April 2025 | Accepted: 19 April 2025 | Online: 27 April 2025


Corresponding author: Abdelhalim Saadi

Abstract

This study evaluates the performance of transformer-based models such as BERT, RoBERTa, and XLNet for fake news detection. Using supervised and unsupervised deep learning techniques, we optimized classification accuracy while reducing computational costs through text summarization. The results show that RoBERTa, fine-tuned with summarized content, achieves 98.39% accuracy, outperforming the other models. Additionally, we assessed AI-generated misinformation using GPT-2, confirming that transformer models effectively distinguish real from synthetic news. We utilized the GPT-2 model instead of more recent models like GPT-4, as our objective was to generate fake news locally and compare it with pretrained models from the same time period.

Keywords:

fake news detection, NLP, DL, transformers, RoBERTa, GPT-2, text classification

Downloads

Download data is not yet available.

References

H. Allcott and M. Gentzkow, "Social Media and Fake News in the 2016 Election," Journal of Economic Perspectives, vol. 31, no. 2, pp. 211–236, May 2017.

M. Potthast, J. Kiesel, K. Reinartz, J. Bevendorff, and B. Stein, "A Stylometric Inquiry into Hyperpartisan and Fake News." arXiv, Feb. 18, 2017.

H. F. Villela, F. Corrêa, J. S. de A. N. Ribeiro, A. Rabelo, and D. B. F. Carvalho, "Fake news detection: a systematic literature review of machine learning algorithms and datasets," Journal on Interactive Systems, vol. 14, no. 1, pp. 47–58, Mar. 2023.

Y. Dou, K. Shu, C. Xia, P. S. Yu, and L. Sun, "User Preference-aware Fake News Detection," in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA, Apr. 2021, pp. 2051–2055.

C. Raffel et al., "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer." arXiv, Sep. 19, 2023.

N. Rai, D. Kumar, N. Kaushik, C. Raj, and A. Ali, "Fake News Classification using transformer based enhanced LSTM and BERT," International Journal of Cognitive Computing in Engineering, vol. 3, pp. 98–105, Jun. 2022.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." arXiv, May 24, 2019.

E. Mustafaraj and P. T. Metaxas, "The Fake News Spreading Plague: Was it Preventable?" arXiv, Mar. 20, 2017.

J. A. Nasir, O. S. Khan, and I. Varlamis, "Fake news detection: A hybrid CNN-RNN based deep learning approach," International Journal of Information Management Data Insights, vol. 1, no. 1, Apr. 2021, Art. no. 100007.

N. K. Conroy, V. L. Rubin, and Y. Chen, "Automatic deception detection: Methods for finding fake news," Proceedings of the Association for Information Science and Technology, vol. 52, no. 1, pp. 1–4, 2015.

S. Kumari and M. P. Singh, "A Deep Learning Multimodal Framework for Fake News Detection," Engineering, Technology & Applied Science Research, vol. 14, no. 5, pp. 16527–16533, Oct. 2024.

A. Vaswani et al., "Attention Is All You Need." arXiv, Aug. 02, 2023.

G0nz4lo-4lvarez-H3rv4s, "G0nz4lo-4lvarez-H3rv4s/FakeNewsDetection." [Online]. Available: https://github.com/G0nz4lo-4lvarez-H3rv4s/FakeNewsDetection.

S. Nagel, "News Dataset Available," Common Crawl, Oct. 04, 2016. https://commoncrawl.org/blog/news-dataset-available.

S. Raza, D. Paulen-Patterson, and C. Ding, "Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data." arXiv, Dec. 20, 2024.

J. Jouhar, A. Pratap, N. Tijo, and M. Mony, "Fake News Detection using Python and Machine Learning," Procedia Computer Science, vol. 233, pp. 763–771, Jan. 2024.

D. Paper, "Introduction to Deep Learning," in TensorFlow 2.x in the Colaboratory Cloud: An Introduction to Deep Learning on Google’s Cloud Service, D. Paper, Ed. Berkeley, CA, USA: Apress, 2021, pp. 1–24.

P. Pookduang, R. Klangbunrueang, W. Chansanam, and T. Lunrasri, "Advancing Sentiment Analysis: Evaluating RoBERTa against Traditional and Deep Learning Models," Engineering, Technology & Applied Science Research, vol. 15, no. 1, pp. 20167–20174, Feb. 2025.

Downloads

How to Cite

[1]
Saadi, A., Belhadef, H., Guessas, A. and Hafirassou, O. 2025. Enhancing Fake News Detection with Transformer Models and Summarization. Engineering, Technology & Applied Science Research. 15, 3 (Jun. 2025), 23253–23259. DOI:https://doi.org/10.48084/etasr.10678.

Metrics

Abstract Views: 141
PDF Downloads: 81

Metrics Information

License

Copyright (c) 2025 Abdelhalim Saadi, Hacene Belhadef, Akram Guessas, Oussama Hafirassou

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.