A Hybrid BERT-CNN with Multihead Self-Attention for Automated Cyberbullying Detection
Received: 14 June 2025 | Revised: 10 October 2025 | Accepted: 18 October 2025 | Online: 4 December 2025
Corresponding author: Daniyar Sultan
Abstract
This paper presents a novel hybrid deep learning architecture, the CustomBERTCNNAttentionModel, designed for the automated detection of cyberbullying in social media text. The proposed model integrates the contextual language understanding capabilities of Bidirectional Encoder Representations from Transformers (BERT) with the local feature extraction strengths of Convolutional Neural Networks (CNNs) and the dynamic relevance weighting of multihead self-attention mechanisms. Evaluated on the Kaggle Cyberbullying Dataset, which includes both binary and multiclass labels, the model demonstrates superior performance compared to traditional classifiers and ensemble methods. The architecture effectively handles imbalanced and noisy text data, achieving an accuracy of 0.9853 in binary classification tasks. A comprehensive evaluation using standard metrics and visual analysis through confusion matrices confirms the model's robustness and its capacity to generalize across diverse types of cyberbullying. These results highlight the effectiveness of combining transformer-based embeddings with attention-enhanced convolutional structures for detecting harmful online behavior and contribute to the advancement of intelligent moderation systems.
Keywords:
cyberbullying detection, Bidirectional Encoder Representations from Transformers (BERT), Convolutional Neural Network (CNN), multihead self-attention, hybrid deep learning, text classification, social media analysis, Natural Language Processing (NLP)Downloads
References
L. M. Al-Harigy, H. A. Al-Nuaim, N. Moradpoor, and Z. Tan, "Building towards Automated Cyberbullying Detection: A Comparative Analysis," Computational Intelligence and Neuroscience, vol. 2022, no. 1, June 2022, Art. no. 4794227. DOI: https://doi.org/10.1155/2022/4794227
N. M. Singh and S. K. Sharma, "An efficient automated multi-modal cyberbullying detection using decision fusion classifier on social media platforms," Multimedia Tools and Applications, vol. 83, no. 7, pp. 20507–20535, Feb. 2024. DOI: https://doi.org/10.1007/s11042-023-16402-w
C. Amol, L. Wanzare, and J. Obuhuma, "Modelling Misinformation in Swahili-English Code-switched Texts," International Journal of Information Technology and Computer Science, vol. 17, no. 1, pp. 67–80, Feb. 2025. DOI: https://doi.org/10.5815/ijitcs.2025.01.05
A. Kumar, R. Sharma, and P. Bedi, "Towards Optimal NLP Solutions: Analyzing GPT and LLaMA-2 Models Across Model Scale, Dataset Size, and Task Diversity," Engineering, Technology & Applied Science Research, vol. 14, no. 3, pp. 14219–14224, June 2024. DOI: https://doi.org/10.48084/etasr.7200
R. Narayan and P. Samanta, "A Machine Learning Approach for Sentiment Analysis Using Social Media Posts," International Journal of Information Technology and Computer Science, vol. 16, no. 5, pp. 23–35, Oct. 2024. DOI: https://doi.org/10.5815/ijitcs.2024.05.02
R. Shamim and M. Lahby, "Automated Detection and Analysis of Cyberbullying Behavior Using Machine Learning," in Combatting Cyberbullying in Digital Media with Artificial Intelligence, 1st ed., M. Lahby, A.-S. K. Pathan, and Y. Maleh, Eds. Boca Raton, FL, USA: Chapman and Hall/CRC, 2023, pp. 116–136. DOI: https://doi.org/10.1201/9781003393061-9
M. Al-Hashedi, L.-K. Soon, H.-N. Goh, A. H. L. Lim, and E.-G. Siew, "Cyberbullying Detection Based on Emotion," IEEE Access, vol. 11, pp. 53907–53918, 2023. DOI: https://doi.org/10.1109/ACCESS.2023.3280556
A. A. Olagunju and I. O. Awoyelu, "Performance Evaluation of Fake News Detection Models," International Journal of Information Technology and Computer Science, vol. 16, no. 6, pp. 89–100, Dec. 2024. DOI: https://doi.org/10.5815/ijitcs.2024.06.07
D. Sultan et al., "A Review of Machine Learning Techniques in Cyberbullying Detection," Computers, Materials & Continua, vol. 74, no. 3, pp. 5625–5640, Dec. 2022. DOI: https://doi.org/10.32604/cmc.2023.033682
S. S. Alzahrani, "Data Mining Regarding Cyberbullying in the Arabic Language on Instagram Using KNIME and Orange Tools," Engineering, Technology & Applied Science Research, vol. 12, no. 5, pp. 9364–9371, Oct. 2022. DOI: https://doi.org/10.48084/etasr.5184
T. H. Teng and K. D. Varathan, "Cyberbullying Detection in Social Networks: A Comparison Between Machine Learning and Transfer Learning Approaches," IEEE Access, vol. 11, pp. 55533–55560, 2023. DOI: https://doi.org/10.1109/ACCESS.2023.3275130
K. S. Ganguly, K. S. Ganguly, and A. Dutta, "A Comparative Study of Statistical (SARIMA) Vis-À-Vis Some Traditional Machine-Learning and Deep-Learning Techniques to Forecast Malaria Incidences in Kolkata of India," International Journal of Information Technology and Computer Science, vol. 17, no. 5, pp. 68–83, Oct. 2025. DOI: https://doi.org/10.5815/ijitcs.2025.05.06
G. Jaradat, M. Shehab, D. Ibrahim, S. Najdawi, and R. Sihwail, "Deep Learning Approaches for Detecting Cyberbullying on Social Media," Journal of Computational and Cognitive Engineering, Mar. 2025. DOI: https://doi.org/10.47852/bonviewJCCE52024162
M. K. Mali et al., "Automatic detection of cyberbullying behaviour on social media using Stacked Bi-Gru attention with BERT model," Expert Systems with Applications, vol. 262, Mar. 2025, Art. no. 125641. DOI: https://doi.org/10.1016/j.eswa.2024.125641
Y. Tashtoush, A. Banysalim, M. Maabreh, S. Al-Eidi, O. Karajeh, and P. Zahariev, "A Deep Learning Framework for Arabic Cyberbullying Detection in Social Networks," Computers, Materials & Continua, vol. 83, no. 2, pp. 3113–3134, Apr. 2025. DOI: https://doi.org/10.32604/cmc.2025.062724
K. I. Arce-Ruelas, O. Alvarez-Xochihua, L. Pellegrin, L. Cardoza-Avendaño, and J. Á. González-Fraga, “Automatic Cyberbullying Detection: a Mexican case in High School and Higher Education students,” IEEE Latin America Transactions, vol. 20, no. 5, pp. 770–779, May 2022. DOI: https://doi.org/10.1109/TLA.2022.9693561
C. Iwendi, G. Srivastava, S. Khan, and P. K. R. Maddikunta, "Cyberbullying detection solutions based on deep learning architectures," Multimedia Systems, vol. 29, no. 3, pp. 1839–1852, June 2023. DOI: https://doi.org/10.1007/s00530-020-00701-5
A. Muneer, A. Alwadain, M. G. Ragab, and A. Alqushaibi, "Cyberbullying Detection on Social Media Using Stacking Ensemble Learning and Enhanced BERT," Information, vol. 14, no. 8, Aug. 2023, Art. no. 467. DOI: https://doi.org/10.3390/info14080467
M. Fahaad Almufareh, N. Zaman Jhanjhi, M. Humayun, G. Naif Alwakid, D. Javed, and S. Naif Almuayqil, "Integrating Sentiment Analysis With Machine Learning for Cyberbullying Detection on Social Media," IEEE Access, vol. 13, pp. 78348–78359, 2025. DOI: https://doi.org/10.1109/ACCESS.2025.3558843
V. Balakrisnan and M. Kaity, "Cyberbullying detection and machine learning: a systematic literature review," Artificial Intelligence Review, vol. 56, no. 1, pp. 1375–1416, Oct. 2023. DOI: https://doi.org/10.1007/s10462-023-10553-w
S. Pericherla and E. Ilavarasan, "Cyberbullying detection and classification on social media images using Convolution Neural Networks and CB-YOLO model," Evolving Systems, vol. 16, no. 2, Feb. 2025, Art. no. 43. DOI: https://doi.org/10.1007/s12530-025-09656-2
A. Al-Marghilani, "Artificial Intelligence-Enabled Cyberbullying-Free Online Social Networks in Smart Cities," International Journal of Computational Intelligence Systems, vol. 15, no. 1, Jan. 2022, Art. no. 9. DOI: https://doi.org/10.1007/s44196-022-00063-y
T. Ahmed, S. Ivan, M. Kabir, H. Mahmud, and K. Hasan, "Performance analysis of transformer-based architectures and their ensembles to detect trait-based cyberbullying," Social Network Analysis and Mining, vol. 12, no. 1, Aug. 2022, Art. no. 99. DOI: https://doi.org/10.1007/s13278-022-00934-4
K. Subhashree and S. M. Kumar, "Enhanced quantum long short-term memory neural network based multi-task learning for sentimental analysis and cyberbullying detection," Expert Systems with Applications, vol. 282, July 2025, Art. no. 127555. DOI: https://doi.org/10.1016/j.eswa.2025.127555
T. Mahmud, M. Ptaszynski, J. Eronen, and F. Masui, "Cyberbullying detection for low-resource languages and dialects: Review of the state of the art," Information Processing & Management, vol. 60, no. 5, Sept. 2023, Art. no. 103454. DOI: https://doi.org/10.1016/j.ipm.2023.103454
D. L. Hall, Y. N. Silva, B. Wheeler, L. Cheng, and K. Baumel, "Harnessing the Power of Interdisciplinary Research with Psychology-Informed Cyberbullying Detection Models," International Journal of Bullying Prevention, vol. 4, no. 1, pp. 47–54, Mar. 2022. DOI: https://doi.org/10.1007/s42380-021-00107-5
M. Al-Ajlan and M. Ykhlef, "Firefly-CDDL: A Firefly-Based Algorithm for Cyberbullying Detection Based on Deep Learning," Computers, Materials & Continua, vol. 75, no. 1, pp. 19–34, Feb. 2023. DOI: https://doi.org/10.32604/cmc.2023.033753
"Cyberbullying Dataset." Kaggle. [Online]. Available: https://www.kaggle.com/datasets/saurabhshahane/cyberbullying-dataset.
B. Ogunleye and B. Dharmaraj, "The Use of a Large Language Model for Cyberbullying Detection," Analytics, vol. 2, no. 3, pp. 694–707, Sept. 2023. DOI: https://doi.org/10.3390/analytics2030038
M. H. Obaid, S. K. Guirguis, and S. M. Elkaffas, "Cyberbullying Detection and Severity Determination Model," IEEE Access, vol. 11, pp. 97391–97399, 2023. DOI: https://doi.org/10.1109/ACCESS.2023.3313113
M. Raj, S. Singh, K. Solanki, and R. Selvanambi, "An Application to Detect Cyberbullying Using Machine Learning and Deep Learning Techniques," SN Computer Science, vol. 3, no. 5, July 2022, Art. no. 401. DOI: https://doi.org/10.1007/s42979-022-01308-5
S. Viswanath and A. K. K M, "Hybrid 1D VGG16 and SVM Framework for Early Detection and Intensity Classification of Cyberbullying," International Journal of Intelligent Engineering and Systems, vol. 18, no. 3, pp. 543–555, Apr. 2025. DOI: https://doi.org/10.22266/ijies2025.0430.37
A. O. Akinwumi, A. O. Ige, J. R. Obafemi, O. D. Akinrolabu, and B. O. Akingbesote, "CBDS-ConvNet: A Cyber-Bullying Detection Model using Convolutional Neural Network," Communications on Applied Electronics, vol. 7, no. 40, pp. 11–21, Jan. 2025. DOI: https://doi.org/10.5120/cae2025652905
Downloads
How to Cite
License
Copyright (c) 2025 Meruyert Yerekesheva, Oxana Akhmetova, Assel Kaziyeva, Daniyar Sultan, Aigerim Toktarova, Rustam Abdrakhmanov, Tolep Abdimukhan
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
