Improving Image Inpainting through Contextual Attention in Deep Learning
Received: 25 March 2024 | Revised: 11 April 2024 | Accepted: 19 April 2024 | Online: 2 August 2024
Corresponding author: Ayoub Charef
Abstract
Image processing is vital in modern technology, offering a diverse range of techniques for manipulating digital images to extract valuable information or enhance visual quality. Among these techniques, image inpainting stands out, involving the reconstruction or restoration of missing or damaged regions within images. This study explores advances in image inpainting and presents a novel approach that integrates coarse-to-fine inpainting and attention-based inpainting techniques. The proposed method leverages deep learning methods to enhance the quality and efficiency of image inpainting, achieving robust and high-quality results that balance structural integrity and contextual coherence. A comprehensive evaluation and comparison with existing methods showed that the proposed approach had superior performance in maintaining structural integrity and contextual coherence within images.
Keywords:
contextual attention, coarse-to-fine inpainting, attention-based inpainting, image processingDownloads
References
Z. Qin, Q. Zeng, Y. Zong, and F. Xu, "Image inpainting based on deep learning: A review," Displays, vol. 69, Sep. 2021, Art. no. 102028. DOI: https://doi.org/10.1016/j.displa.2021.102028
O. Elharrouss, N. Almaadeed, S. Al-Maadeed, and Y. Akbari, "Image Inpainting: A Review," Neural Processing Letters, vol. 51, no. 2, pp. 2007–2028, Apr. 2020. DOI: https://doi.org/10.1007/s11063-019-10163-0
R. H. Mwawado, B. J. Maiseli, and M. A. Dida, "Robust Edge Detection Method for Segmentation of Diabetic Foot Ulcer Images," Engineering, Technology & Applied Science Research, vol. 10, no. 4, pp. 6034–6040, Aug. 2020. DOI: https://doi.org/10.48084/etasr.3495
A. Pajot, E. de Bezenac, and P. Gallinari, "Unsupervised Adversarial Image Inpainting." arXiv, Dec. 18, 2019.
L. Zhao et al., "UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun. 2020, pp. 5740–5749. DOI: https://doi.org/10.1109/CVPR42600.2020.00578
C. T. Li, W. C. Siu, Z. S. Liu, L. W. Wang, and D. P. K. Lun, "DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting," in Computer Vision – ECCV 2020 Workshops, Glasgow, UK, Aug. 2020, pp. 5–22. DOI: https://doi.org/10.1007/978-3-030-66823-5_1
T. Yu et al., "Region Normalization for Image Inpainting," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 12733–12740, Apr. 2020. DOI: https://doi.org/10.1609/aaai.v34i07.6967
A. Charef, Z. Jarir, and M. Quafafou, "Smart System for Emergency Traffic Recommendations: Urban Ambulance Mobility," International Journal of Advanced Computer Science and Applications, vol. 13, no. 10, pp. 32–45, 2022. DOI: https://doi.org/10.14569/IJACSA.2022.0131005
W. Xiong et al., “Foreground-Aware Image Inpainting,” presented at the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, Jun. 2019, pp. 5833–5841. DOI: https://doi.org/10.1109/CVPR.2019.00599
J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, and T. Huang, "Free-Form Image Inpainting With Gated Convolution," in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Oct. 2019, pp. 4470–4479. DOI: https://doi.org/10.1109/ICCV.2019.00457
Y. Zeng, J. Fu, H. Chao, and B. Guo, "Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, Jun. 2019, pp. 1486–1494. DOI: https://doi.org/10.1109/CVPR.2019.00158
Y. Zeng, J. Fu, H. Chao, and B. Guo, "Aggregated Contextual Transformations for High-Resolution Image Inpainting," IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 7, pp. 3266–3280, Mar. 2023. DOI: https://doi.org/10.1109/TVCG.2022.3156949
H. Liu, Z. Wan, W. Huang, Y. Song, X. Han, and J. Liao, "PD-GAN: Probabilistic Diverse GAN for Image Inpainting," in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, Jun. 2021, pp. 9367–9376. DOI: https://doi.org/10.1109/CVPR46437.2021.00925
Y. Chen et al., "The improved image inpainting algorithm via encoder and similarity constraint," The Visual Computer, vol. 37, no. 7, pp. 1691–1705, Jul. 2021. DOI: https://doi.org/10.1007/s00371-020-01932-3
Z. Yi, Q. Tang, S. Azizi, D. Jang, and Z. Xu, "Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun. 2020, pp. 7505–7514. DOI: https://doi.org/10.1109/CVPR42600.2020.00753
D. J. B. Rojas, B. J. T. Fernandes, and S. M. M. Fernandes, "A Review on Image Inpainting Techniques and Datasets," in 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Porto de Galinhas, Brazil, Nov. 2020, pp. 240–247. DOI: https://doi.org/10.1109/SIBGRAPI51738.2020.00040
V. Yatnalli, B. G. Shivaleelavathi, and K. L. Sudha, "Review of Inpainting Algorithms for Wireless Communication Application," Engineering, Technology & Applied Science Research, vol. 10, no. 3, pp. 5790–5795, Jun. 2020. DOI: https://doi.org/10.48084/etasr.3547
J. Li, F. He, L. Zhang, B. Du, and D. Tao, "Progressive Reconstruction of Visual Structure for Image Inpainting," in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Oct. 2019, pp. 5961–5970. DOI: https://doi.org/10.1109/ICCV.2019.00606
Downloads
How to Cite
License
Copyright (c) 2024 Ayoub Charef, Ahmed Ouqour
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
