An Improved Medical Image Fusion for Efficient Diagnostic Imaging Using Deep Learning and Gradient-Based Quality Assessment

Walid El-Shafai; Ahmad Taher Azar

doi:10.48084/etasr.10622

Authors

Walid El-Shafai College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia | Automated Systems and Computing Lab (ASCL), Prince Sultan University, Riyadh, Saudi Arabia | Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menoufia 32952, Egypt
Ahmad Taher Azar College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia | Automated Systems and Computing Lab (ASCL), Prince Sultan University, Riyadh, Saudi Arabia

Volume: 16 | Issue: 1 | Pages: 31066-31075 | February 2026 | https://doi.org/10.48084/etasr.10622

Received: 18 February 2025 | Revised: 4 May 2025 and 25 May 2025 | Accepted: 30 May 2025 | Online: 1 January 2026

Corresponding author: Ahmad Taher Azar

Abstract

Advancements in Deep Learning (DL) have significantly contributed to progress in image processing and pattern recognition, particularly in the medical imaging domain. This study proposes an improved DL-based fusion method for multi-modal medical images, referred to as Deep Learning Medical Image Fusion (DMF). The proposed technique integrates Convolutional Neural Networks (CNNs) with three conventional image fusion methods to generate enhanced fused outputs across various medical imaging modalities. By leveraging gradient-based quality assessments, including visibility, consistency, and exposure mapping, the DMF method effectively synthesizes multi-exposure images, resulting in high-resolution outputs with improved structural clarity and visual consistency. Unlike traditional fusion approaches, the proposed model capitalizes on gradient variations among multiple exposures to guide fusion decisions, leading to improved diagnostic relevance. The performance of the DMF technique is validated through comprehensive experiments, involving Magnetic Resonance Imaging-Computed Tomography (MRI-CT) and MRI-Positron Emission Tomography (MRI-PET) image pairs, which demonstrated its superior effectiveness and practicality in healthcare imaging and diagnostic applications.

Keywords:

medical image fusion, deep learning, multi-modal imaging, convolutional neural networks, gradient-based quality assessment, image processing, diagnostic imaging, healthcare applications

Downloads

Download data is not yet available.

References

S. U. Khan, I. Ullah, N. Ullah, S. Shah, M. E. Affendi, and B. Lee, "A Novel CT Image De-Noising and Fusion Based Deep Learning Network to Screen for Disease (COVID-19)," Scientific Reports, vol. 13, no. 1, Apr. 2023, Art. no. 6601. DOI: https://doi.org/10.1038/s41598-023-33614-0

S. M. Fati, E. M. Senan, and N. ElHakim, "Deep and Hybrid Learning Technique for Early Detection of Tuberculosis Based on X-ray Images Using Feature Fusion," Applied Sciences, vol. 12, no. 14, July 2022, Art. no. 7092. DOI: https://doi.org/10.3390/app12147092

R. K. Jogi and M. U. Rani, "A Review of Multimodal Medical Imaging Fusion Methods," GAMANAM: Global Advances in Multidisciplinary Applications in Next-Gen And Modern Technologies, vol. 1, no. 2, pp. 90–101, Apr. 2025.

N. Wickramasinghe, B. R. Thompson, and J. Xiao, "The Opportunities and Challenges of Digital Anatomy for Medical Sciences: Narrative Review," JMIR Medical Education, vol. 8, no. 2, May 2022, Art. no. e34687. DOI: https://doi.org/10.2196/34687

W. Tang, Y. Liu, J. Cheng, C. Li, and X. Chen, "Green Fluorescent Protein and Phase Contrast Image Fusion Via Detail Preserving Cross Network," IEEE Transactions on Computational Imaging, vol. 7, pp. 584–597, May 2021. DOI: https://doi.org/10.1109/TCI.2021.3083965

S. Karim, G. Tong, J. Li, A. Qadir, U. Farooq, and Y. Yu, "Current Advances and Future Perspectives of Image Fusion: A Comprehensive Review," Information Fusion, vol. 90, pp. 185–217, Feb. 2023. DOI: https://doi.org/10.1016/j.inffus.2022.09.019

C. Ghandour, W. El-Shafai, and S. El-Rabaie, "Medical Image Fusion Based on Weighted Least Square Optimization and Deep Learning Algorithm," in 2021 9th International Japan-Africa Conference on Electronics, Communications, and Computations, Alexandria, Egypt, Dec. 2021, pp. 159–163. DOI: https://doi.org/10.1109/JAC-ECC54461.2021.9691453

S. Singh et al., "A Review of Image Fusion: Methods, Applications and Performance Metrics," Digital Signal Processing, vol. 137, June 2023, Art. no. 104020. DOI: https://doi.org/10.1016/j.dsp.2023.104020

Z. Wang, Y. Ma, and Y. Zhang, "Review of Pixel-Level Remote Sensing Image Fusion Based on Deep Learning," Information Fusion, vol. 90, pp. 36–58, Feb. 2023. DOI: https://doi.org/10.1016/j.inffus.2022.09.008

N. A. El‐Hag et al., "Utilization of Image Interpolation and Fusion in Brain Tumor Segmentation," International Journal for Numerical Methods in Biomedical Engineering, vol. 37, no. 8, Aug. 2021, Art. no. e3449. DOI: https://doi.org/10.1002/cnm.3449

X. Liu, L. Jiao, L. Li, X. Tang, and Y. Guo, "Deep Multi-Level Fusion Network for Multi-Source Image Pixel-Wise Classification," Knowledge-Based Systems, vol. 221, June 2021, Art. no. 106921. DOI: https://doi.org/10.1016/j.knosys.2021.106921

M. Yin, X. Liu, Y. Liu, and X. Chen, "Medical Image Fusion With Parameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain," IEEE Transactions on Instrumentation and Measurement, vol. 68, no. 1, pp. 49–64, Jan. 2019. DOI: https://doi.org/10.1109/TIM.2018.2838778

P. Ganasala and A. D. Prasad, "Medical Image Fusion Based on Laws of Texture Energy Measures in Stationary Wavelet Transform Domain," International Journal of Imaging Systems and Technology, vol. 30, no. 3, pp. 544–557, Sept. 2020. DOI: https://doi.org/10.1002/ima.22393

H. Inbarani, A. T. Azar, and G. Jothi, "Leukemia Image Segmentation Using a Hybrid Histogram-Based Soft Covering Rough K-Means Clustering Algorithm," Electronics, vol. 9, no. 1, Jan. 2020, Art. no. 188. DOI: https://doi.org/10.3390/electronics9010188

A. D. Prasad and P. Ganasala, "MRI and SPECT Image Fusion Based on Adaptive Fusion Rule in Transform Domain," in Intelligent Computing Techniques in Biomedical Imaging. Amsterdam, Netherlands: Elsevier, 2025, pp. 281–293. DOI: https://doi.org/10.1016/B978-0-443-15999-2.00001-3

M. Diwakar et al., "Multimodality Medical Image Fusion Using Clustered Dictionary Learning in Non-Subsampled Shearlet Transform," Diagnostics, vol. 13, no. 8, Apr. 2023, Art. no. 1395. DOI: https://doi.org/10.3390/diagnostics13081395

D. K. Sharma, J. Sachdeva, C. K. Ahuja, and A. Singh, "A Novel Hybrid Multimodal Medical Image Fusion Scheme Based on Non-subsampled Shearlet Transform," Circuits, Systems, and Signal Processing, vol. 43, no. 6, pp. 3627–3648, June 2024. DOI: https://doi.org/10.1007/s00034-024-02614-4

H. Zhang, H. Xu, X. Tian, J. Jiang, and J. Ma, "Image Fusion Meets Deep Learning: A Survey and Perspective," Information Fusion, vol. 76, pp. 323–336, Dec. 2021. DOI: https://doi.org/10.1016/j.inffus.2021.06.008

S. M. Fati, E. M. Senan, and A. T. Azar, "Hybrid and Deep Learning Approach for Early Diagnosis of Lower Gastrointestinal Diseases," Sensors, vol. 22, no. 11, May 2022, Art. no. 4079. DOI: https://doi.org/10.3390/s22114079

A. Abdulrahman Albraikan, N. Nemri, M. Abdullah Alkhonaini, A. Mustafa Hilal, I. Yaseen, and A. Motwakel, "Automated Deep Learning Based Melanoma Detection and Classification Using Biomedical Dermoscopic Images," Computers, Materials & Continua, vol. 74, no. 2, pp. 2443–2459, 2023. DOI: https://doi.org/10.32604/cmc.2023.026379

Y. Liu, L. Wang, H. Li, and X. Chen, "Multi-Focus Image Fusion with Deep Residual Learning and Focus Property Detection," Information Fusion, vol. 86–87, pp. 1–16, Oct. 2022. DOI: https://doi.org/10.1016/j.inffus.2022.06.001

K. Wang, M. Zheng, H. Wei, G. Qi, and Y. Li, "Multi-Modality Medical Image Fusion Using Convolutional Neural Network and Contrast Pyramid," Sensors, vol. 20, no. 8, Apr. 2020, Art. no. 2169. DOI: https://doi.org/10.3390/s20082169

W. Kong, C. Li, and Y. Lei, "Multimodal Medical Image Fusion using Convolutional Neural Network and Extreme Learning Machine," Frontiers in Neurorobotics, vol. 16, Nov. 2022, Art. no. 1050981. DOI: https://doi.org/10.3389/fnbot.2022.1050981

T. Du et al., "Overcoming the Challenges of Multi-Modal Medical Image Sharing: A Novel Data Distillation Strategy via Contrastive Learning," Neurocomputing, vol. 617, Feb. 2025, Art. no. 129043. DOI: https://doi.org/10.1016/j.neucom.2024.129043

K. Koteswara Rao and K. Veera Swamy, "Multimodal Medical Image Fusion Using the MBM-PCNN Model," IETE Journal of Research, vol. 71, no. 6, pp. 2161–2175, June 2025. DOI: https://doi.org/10.1080/03772063.2025.2477746

B. K. Shreyamsha Kumar, "Image Fusion Based on Pixel Significance using Cross Bilateral Filter," Signal, Image and Video Processing, vol. 9, no. 5, pp. 1193–1204, July 2015. DOI: https://doi.org/10.1007/s11760-013-0556-9

W. Xu, H. Ding, L. Deng, C. Liu, X. Lu, and L. Zhu, "Infrared and Visible Image Fusion Based on Fourth Order Partial Differential Equation and Cross Bilateral Filter," in Eleventh International Symposium on Precision Mechanical Measurements, Guangzhou, China, Sept. 2024, Art. no. 17. DOI: https://doi.org/10.1117/12.3032399

Y. Zhou, X. Yang, S. Liu, and J. Yin, "Multimodal Medical Image Fusion Network Based on Target Information Enhancement," IEEE Access, vol. 12, pp. 70851–70869, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3402965

L. Ren, Z. Pan, J. Cao, J. Liao, and Y. Wang, "Infrared and Visible Image Fusion Based on Weighted Variance Guided Filter and Image Contrast Enhancement," Infrared Physics & Technology, vol. 114, May 2021, Art. no. 103662. DOI: https://doi.org/10.1016/j.infrared.2021.103662

S. Budhiraja, S. Agrawal, and N. Sharma, "Infrared and Visible Image Fusion based on Sparse Representation and Weighted Least Square Optimization," IETE Journal of Research, vol. 71, no. 5, pp. 1504–1516, May 2025. DOI: https://doi.org/10.1080/03772063.2025.2469642

X. Han et al., "An Adaptive Two-Scale Image Fusion of Visible and Infrared Images," IEEE Access, vol. 7, pp. 56341–56352, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2913289

Y. Yang et al., "LatLRR-CNN: An Infrared and Visible Image Fusion Method Combining Latent Low-Rank Representation and CNN," Multimedia Tools and Applications, vol. 82, no. 23, pp. 36303–36323, Sept. 2023. DOI: https://doi.org/10.1007/s11042-023-14967-0

M. M. Nanavati and M. Shah, "Implementation and Comparative Study of Pyramid-based Image Fusion Techniques for Lumbar Spine Images," Engineering, Technology & Applied Science Research, vol. 13, no. 4, pp. 11139–11145, Aug. 2023. DOI: https://doi.org/10.48084/etasr.5960

X. Zeng, Z. Huang, L. Xu, and Y. Xie, "CP‐GAN: Meet the High Requirements of Diagnose Report to Medical Image By Content Preservation," IET Image Processing, vol. 16, no. 1, pp. 29–38, Jan. 2022. DOI: https://doi.org/10.1049/ipr2.12145

N. Kumar, A. Hashmi, M. Gupta, and A. Kundu, "Automatic Diagnosis of Covid-19 Related Pneumonia from CXR and CT-Scan Images," Engineering, Technology & Applied Science Research, vol. 12, no. 1, pp. 7993–7997, Feb. 2022. DOI: https://doi.org/10.48084/etasr.4613

K. Clark et al., "The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository," Journal of Digital Imaging, vol. 26, no. 6, pp. 1045–1057, Dec. 2013. DOI: https://doi.org/10.1007/s10278-013-9622-7

C. R. Jack et al., "The Alzheimer's Disease Neuroimaging Initiative (ADNI): MRI Methods," Journal of Magnetic Resonance Imaging, vol. 27, no. 4, pp. 685–691, Apr. 2008. DOI: https://doi.org/10.1002/jmri.21049