Advancing Sentiment Analysis: Evaluating RoBERTa against Traditional and Deep Learning Models

Authors

  • Pongsathon Pookduang Department of Information Science, Faculty of Humanities and Social Sciences, Khon Kaen University, Thailand https://orcid.org/0009-0007-1359-1195
  • Rapeepat Klangbunrueang Department of Information Science, Faculty of Humanities and Social Sciences, Khon Kaen University, Thailand https://orcid.org/0009-0006-0869-6614
  • Wirapong Chansanam Department of Information Science, Faculty of Humanities and Social Sciences, Khon Kaen University, Thailand
  • Tassanee Lunrasri Department of Information Systems, Faculty of Business Administration and Information Technology, Rajamangala University of Technology, Khon Kaen, Thailand https://orcid.org/0009-0006-7211-7226
Volume: 15 | Issue: 1 | Pages: 20167-20174 | February 2025 | https://doi.org/10.48084/etasr.9703

Abstract

This research evaluates the performance of various sentiment analysis models, including traditional machine learning approaches (Naive Bayes, KNN, CART), a deep learning model (LSTM), and the transformer-based model RoBERTa using an Amazon book reviews dataset. ROBERTa outperformed all other models, achieving an accuracy of 96.30% and an F1-score of 98.11%, underscoring its superior ability to process complex and semantically diverse textual data. Traditional models, while computationally efficient, demonstrated limitations in capturing nuanced textual relationships, and the LSTM model, although competitive, faced scalability challenges and overfitting issues. These results demonstrate how transformer-based architectures such as RoBERTa offer advantages in real-world applications, particularly in e-commerce and social media sentiment analysis. This study underscores the superior capabilities of RoBERTa for sentiment analysis, particularly in processing semantically diverse and context-rich textual data that traditional models struggle to capture. Future work will explore optimizing RoBERTa's computational efficiency and expanding its applications to multilingual and cross-domain sentiment analysis tasks.

Keywords:

sentiment analysis, RoBERTa, Amazon book reviews, deep learning, machine learning model

Downloads

Download data is not yet available.

References

A. Alsaeedi and M. Zubair, "A Study on Sentiment Analysis Techniques of Twitter Data," International Journal of Advanced Computer Science and Applications, vol. 10, no. 2, 2019.

W. Chansanam and K. Tuamsuk, "Thai Twitter Sentiment Analysis: Performance Monitoring of Politics in Thailand using Text Mining Techniques," International Journal of Innovation, vol. 11, no. 12, 2020.

S. Sweta, "Application of Sentiment Analysis in Diverse Domains," in Sentiment Analysis and its Application in Educational Data Mining, S. Sweta, Ed. Singapore: Springer Nature, 2024, pp. 19–46.

N. Shrestha and F. Nasoz, "Deep Learning Sentiment Analysis of Amazon.com Reviews and Ratings," International Journal on Soft Computing, Artificial Intelligence and Applications, vol. 8, no. 1, pp. 01–15, Feb. 2019.

K. L. Tan, C. P. Lee, K. S. M. Anbananthen, and K. M. Lim, "RoBERTa-LSTM: A Hybrid Model for Sentiment Analysis With Transformer and Recurrent Neural Network," IEEE Access, vol. 10, pp. 21517–21525, 2022.

A. S. Lv, D. Babu.M, A.Manonmani, Y. M. Reeja, M. S. S, and A. R. Kumar, "An Efficient Approach in Selection of Information-Gaining Features Using Sentiment Analysis," Journal of Computational Analysis and Applications (JoCAAA), vol. 33, no. 05, pp. 719–725, Sep. 2024.

J. M. T. Habib and A. A. Poguda, "Comparison of Deep Learning Sentiment Analysis Methods, Including LSTM and Machine Learning," Open Education, vol. 27, no. 4, pp. 60–71, Aug. 2023.

K. L. Tan, C. P. Lee, and K. M. Lim, "RoBERTa-GRU: A Hybrid Deep Learning Model for Enhanced Sentiment Analysis," Applied Sciences, vol. 13, no. 6, Jan. 2023, Art. no. 3915.

M. M. Rahman, A. I. Shiplu, Y. Watanobe, and M. A. Alam, "RoBERTa-BiLSTM: A Context-Aware Hybrid Model for Sentiment Analysis." arXiv, Jun. 01, 2024.

C. Aspillaga, A. Carvallo, and V. Araujo, "Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks." arXiv, Mar. 27, 2020.

A. Rawat, H. Maheshwari, M. Khanduja, R. Kumar, M. Memoria, and S. Kumar, "Sentiment Analysis of Covid19 Vaccines Tweets Using NLP and Machine Learning Classifiers," in 2022 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COM-IT-CON), Faridabad, India, May 2022, pp. 225–230.

O. Iparraguirre-Villanueva et al., "The Public Health Contribution of Sentiment Analysis of Monkeypox Tweets to Detect Polarities Using the CNN-LSTM Model," Vaccines, vol. 11, no. 2, Feb. 2023, Art. no. 312.

K. K. Mohbey, G. Meena, S. Kumar, and K. Lokesh, "A CNN-LSTM-Based Hybrid Deep Learning Approach for Sentiment Analysis on Monkeypox Tweets," New Generation Computing, vol. 42, no. 1, pp. 89–107, Mar. 2024.

A. K. Laturiuw and Y. A. Singgalen, "Sentiment Analysis of Raja Ampat Tourism Destination Using CRISP-DM: SVM, NBC, DT, and k-NN Algorithm," Journal of Information Systems and Informatics, vol. 5, no. 2, pp. 518–535, May 2023.

D. Suryadi and J. T. Sabarman, "Analyzing Restaurants in Tourism Destinations Through Online Reviews Using Topic Modeling and Sentiment Analysis," in 2023 IEEE 9th Information Technology International Seminar (ITIS), Batu Malang, Indonesia, Oct. 2023, pp. 1–6.

"Amazon Books Reviews." Kaggle, Accessed: Jan. 04, 2025. [Online]. Available: https://www.kaggle.com/datasets/mohamedbakhet/amazon-books-reviews.

A. McCallum and K. Nigam, "A comparison of event models for naive bayes text classification," in AAAI-98 workshop on learning for text categorization, 1998, vol. 752, no. 1, pp. 41–48.

C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval, 1st ed. Cambridge University Press, 2008.

D. D. Lewis, "Naive (Bayes) at forty: The independence assumption in information retrieval," in Machine Learning: ECML-98, 1998, pp. 4–15.

S. L. Salzberg, "C4.5: Programs for Machine Learning," Machine Learning, vol. 16, no. 3, pp. 235–240, Sep. 1994.

L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification And Regression Trees, 1st ed. Routledge, 2017.

T. Cover and P. Hart, "Nearest neighbor pattern classification," IEEE Transactions on Information Theory, vol. 13, no. 1, pp. 21–27, Jan. 1967.

N. S. Altman, "An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression," The American Statistician, vol. 46, no. 3, pp. 175–185, Aug. 1992.

L. Peterson, "K-nearest neighbor," Scholarpedia, vol. 4, no. 2, 2009, Art. no. 1883.

S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997.

F. A. Gers, J. Schmidhuber, and F. Cummins, "Learning to Forget: Continual Prediction with LSTM," Neural Computation, vol. 12, no. 10, pp. 2451–2471, Oct. 2000.

A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM networks," in Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., Montreal, Canada, 2005, vol. 4, pp. 2047–2052.

Y. Liu et al., "RoBERTa: A Robustly Optimized BERT Pretraining Approach." arXiv, Jul. 26, 2019.

J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." arXiv, May 24, 2019.

Z. Sun, H. Yu, X. Song, R. Liu, Y. Yang, and D. Zhou, "MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices," in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, 2020, pp. 2158–2170.

D. M. W. Powers, "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation." arXiv, Oct. 11, 2020.

M. Sokolova and G. Lapalme, "A systematic analysis of performance measures for classification tasks," Information Processing & Management, vol. 45, no. 4, pp. 427–437, Jul. 2009.

H. M and S. M.N, "A Review on Evaluation Metrics for Data Classification Evaluations," International Journal of Data Mining & Knowledge Management Process, vol. 5, no. 2, pp. 01–11, Mar. 2015.

T. Ngootip, "Enhancing Network Intrusion Detection in Cloud Computing Using a Deep Boltzmann Machine and LightGBM Ensemble Model: A Performance Evaluation on the NSL-KDD Dataset," Sociolytics Journal, vol. 1, no. 1, pp. 1–7, Sep. 2024.

S. Pansayta and W. Chansanam, "Thai COVID-19 patient clustering for monitoring and prevention: data mining techniques," IAES International Journal of Artificial Intelligence (IJ-AI), vol. 13, no. 1, Mar. 2024, Art. no. 256.

P. Manorom, U. Detthamrong, and W. Chansanam, "Comparative Assessment of Fraudulent Financial Transactions using the Machine Learning Algorithms Decision Tree, Logistic Regression, Naïve Bayes, K-Nearest Neighbor, and Random Forest," Engineering, Technology & Applied Science Research, vol. 14, no. 4, pp. 15676–15680, Aug. 2024.

M. K. Myee, R. D. C. Rebekah, T. Deepa, G. D. Zion, and K. Lokesh, "Detection of Depression in Social Media Posts using Emotional Intensity Analysis," Engineering, Technology & Applied Science Research, vol. 14, no. 5, pp. 16207–16211, Oct. 2024.

Downloads

How to Cite

[1]
Pookduang, P., Klangbunrueang, R., Chansanam, W. and Lunrasri, T. 2025. Advancing Sentiment Analysis: Evaluating RoBERTa against Traditional and Deep Learning Models. Engineering, Technology & Applied Science Research. 15, 1 (Feb. 2025), 20167–20174. DOI:https://doi.org/10.48084/etasr.9703.

Metrics

Abstract Views: 120
PDF Downloads: 49

Metrics Information

Most read articles by the same author(s)