Combining Local and Global Feature Extraction for Brain Tumor Classification: A Vision Transformer and iResNet Hybrid Model
Received: 30 June 2024 | Revised: 24 July 2024, 8 August 2024, and 14 August 2024 | Accepted: 18 August 2024 | Online: 23 August 2024
Corresponding author: Amar Y. Jaffar
Abstract
Early diagnosis of brain tumors is crucial for effective treatment and patient prognosis. Traditional Convolutional Neural Networks (CNNs) have shown promise in medical imaging but have limitations in capturing long-range dependencies and contextual information. Vision Transformers (ViTs) address these limitations by leveraging self-attention mechanisms to capture both local and global features. This study aims to enhance brain tumor classification by integrating an improved ResNet (iResNet) architecture with a ViT, creating a robust hybrid model that combines the local feature extraction capabilities of iResNet with the global feature extraction strengths of ViTs. This integration results in a significant improvement in classification accuracy, achieving an overall accuracy of 99.2%, outperforming established models such as InceptionV3, ResNet, and DenseNet. High precision, recall, and F1 scores were observed across all tumor classes, demonstrating the model's robustness and reliability. The significance of the proposed method lies in its ability to effectively capture both local and global features, leading to superior performance in brain tumor classification. This approach offers a powerful tool for clinical decision-making, improving early detection and treatment planning, ultimately contributing to better patient outcomes.
Keywords:
brain tumor classification, vision transformers, iResNet, MRI, deep learning, artificial intelligence, medical imagingDownloads
References
T. Gupta et al., "Comparison of Epidemiology and Outcomes in Neuro-Oncology Between the East and the West: Challenges and Opportunities," Clinical Oncology, vol. 31, no. 8, pp. 539–548, Aug. 2019.
H. Sung et al., "Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries," CA: A Cancer Journal for Clinicians, vol. 71, no. 3, pp. 209–249, 2021.
H. A. Jalab and A. M. Hasan, "Magnetic Resonance Imaging Segmentation Techniques of Brain Tumors: A Review," Archives of Neuroscience, vol. 6, Jan. 2019, Art. no. e84920.
V. Vidhya et al., "Automated Detection and Screening of Traumatic Brain Injury (TBI) Using Computed Tomography Images: A Comprehensive Review and Future Perspectives," International Journal of Environmental Research and Public Health, vol. 18, no. 12, Jan. 2021, Art. no. 6499.
A. Krauze, Y. Zhuge, R. Zhao, E. Tasci, and K. Camphausen, "AI-Driven Image Analysis in Central Nervous System Tumors-Traditional Machine Learning, Deep Learning and Hybrid Models," Journal of biotechnology and biomedicine, vol. 5, no. 1, pp. 1–19, 2022.
T. Imran, A. S. Alghamdi, and M. S. Alkatheiri, "Enhanced Skin Cancer Classification using Deep Learning and Nature-based Feature Optimization," Engineering, Technology & Applied Science Research, vol. 14, no. 1, pp. 12702–12710, Feb. 2024.
S. Tripathi, S. K. Singh, and H. K. Lee, "An end-to-end breast tumour classification model using context-based patch modelling – A BiLSTM approach for image classification," Computerized Medical Imaging and Graphics, vol. 87, Jan. 2021, Art. no. 101838.
S. Singh, M. Kumar, A. Kumar, B. K. Verma, K. Abhishek, and S. Selvarajan, "Efficient pneumonia detection using Vision Transformers on chest X-rays," Scientific Reports, vol. 14, no. 1, Jan. 2024, Art. no. 2487.
M. Talo, O. Yildirim, U. B. Baloglu, G. Aydin, and U. R. Acharya, "Convolutional neural networks for multi-class brain disease detection using MRI images," Computerized Medical Imaging and Graphics, vol. 78, Dec. 2019, Art. no. 101673.
H. Bingol and B. Alatas, "Classification of Brain Tumor Images using Deep Learning Methods," Turkish Journal of Science and Technology, vol. 16, no. 1, pp. 137–143, Mar. 2021.
R. Mehrotra, M. A. Ansari, R. Agrawal, and R. S. Anand, "A Transfer Learning approach for AI-based classification of brain tumors," Machine Learning with Applications, vol. 2, Dec. 2020, Art. no. 100003.
M. Aamir et al., "A deep learning approach for brain tumor classification using MRI images," Computers and Electrical Engineering, vol. 101, Jul. 2022, Art. no. 108105.
W. Ayadi, W. Elhamzi, I. Charfi, and M. Atri, "Deep CNN for Brain Tumor Classification," Neural Processing Letters, vol. 53, no. 1, pp. 671–700, Feb. 2021.
X. Li, H. Chen, X. Qi, Q. Dou, C. W. Fu, and P. A. Heng, "H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes," IEEE Transactions on Medical Imaging, vol. 37, no. 12, pp. 2663–2674, Sep. 2018.
J. S. Paul, A. J. Plassard, B. A. Landman, and D. Fabbri, "Deep learning for brain tumor classification," in Medical Imaging 2017: Biomedical Applications in Molecular, Structural, and Functional Imaging, Mar. 2017, vol. 10137, pp. 253–268.
G. Wang et al., "Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning," IEEE Transactions on Medical Imaging, vol. 37, no. 7, pp. 1562–1573, Jul. 2018.
N. Abiwinanda, M. Hanif, S. T. Hesaputra, A. Handayani, and T. R. Mengko, "Brain Tumor Classification Using Convolutional Neural Network," in World Congress on Medical Physics and Biomedical Engineering 2018, Prague, Czech Republic, 2019, pp. 183–189.
Z. Zhong, M. Zheng, H. Mai, J. Zhao, and X. Liu, "Cancer image classification based on DenseNet model," Journal of Physics: Conference Series, vol. 1651, no. 1, Aug. 2020, Art. no. 012143.
P. Afshar, A. Mohammadi, and K. N. Plataniotis, "Brain Tumor Type Classification via Capsule Networks," in 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, Jul. 2018, pp. 3129–3133.
F. Isensee, P. F. Jaeger, S. A. A. Kohl, J. Petersen, and K. H. Maier-Hein, "nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation," Nature Methods, vol. 18, no. 2, pp. 203–211, Feb. 2021.
A. Sekhar, S. Biswas, R. Hazra, A. K. Sunaniya, A. Mukherjee, and L. Yang, "Brain Tumor Classification Using Fine-Tuned GoogLeNet Features and Machine Learning Algorithms: IoMT Enabled CAD System," IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 3, pp. 983–991, Mar. 2022.
S. Kuraparthi et al., "Brain Tumor Classification of MRI Images Using Deep Convolutional Neural Network," Traitement du Signal, vol. 38, no. 4, pp. 1171–1179, Aug. 2021.
S. Tummala, S. Kadry, S. A. C. Bukhari, and H. T. Rauf, "Classification of Brain Tumor from Magnetic Resonance Imaging Using Vision Transformers Ensembling," Current Oncology, vol. 29, no. 10, pp. 7498–7511, Oct. 2022.
X. Xu and P. Prasanna, "Brain Cancer Survival Prediction on Treatment-Naïve MRI using Deep Anchor Attention Learning with Vision Transformer," in 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), Mar. 2022, pp. 1–5.
S. Nallamolu, H. Nandanwar, A. Singh, and C. N. Subalalitha, "A CNN-based Approach for Multi-Classification of Brain Tumors," in 2022 2nd Asian Conference on Innovation in Technology (ASIANCON), Ravet, India, Aug. 2022, pp. 1–6.
J. P. Dou et al., "Research progress of quantum memory," Acta Physica Sinica, vol. 68, no. 3, 2019, Art. no. 030307.
M. F. I. Soumik and M. A. Hossain, "Brain Tumor Classification With Inception Network Based Deep Learning Model Using Transfer Learning," in 2020 IEEE Region 10 Symposium (TENSYMP), Dhaka, Bangladesh, Jun. 2020, pp. 1018–1021.
Ö. Polat and C. Güngen, "Classification of brain tumors from MR images using deep transfer learning," The Journal of Supercomputing, vol. 77, no. 7, pp. 7236–7252, Jul. 2021.
A. Raza, M. S. Alshehri, S. Almakdi, A. A. Siddique, M. Alsulami, and M. Alhaisoni, "Enhancing brain tumor classification with transfer learning: Leveraging DenseNet121 for accurate and efficient detection," International Journal of Imaging Systems and Technology, vol. 34, no. 1, 2024, Art. no. e22957.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, pp. 770–778.
J. Cheng, "Brain tumor dataset." Figshare, 2017.
Downloads
How to Cite
License
Copyright (c) 2024 Amar Y. Jaffar
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
