Context-Aware Code Summarization Using Multimodal Transformer - Saif, Sadia; Yaseen, Muhammad; Khattak, Umar Farooq; Rahman, Gohar (Engineering, Technology & Applied Science Research, 2026)
Modern software systems continue to grow in complexity, making it increasingly challenging for developers to understand code without clear and up-to-date documentation. This study proposes a multimodal transformer architecture based on CodeT5, enhanced with Abstract Syntax Tree (AST) information to improve both code summarization and the detection of semantic bugs. The suggested framework is designed to capture token-level, structural, and contextual cues, enabling deeper program comprehension than traditional text-only models. The model has been trained and tested on the CoNaLa dataset and compared to baseline and CodeT5 models. Experimental results show substantial improvements, achieving a Bilingual Evaluation Understudy (BLEU) score of 81.34 (an improvement of 44.14 points over CodeT5) and a Recall-Oriented Understudy for Gisting Evaluation–Longest Common Subsequence (ROUGE-L) score of 0.89. These findings confirm that incorporating structural awareness significantly enhances summary relevance and bug-identification capability. The study contributes a scalable, context-sensitive model for automated software understanding and offers strong potential for integration into real-world development tools.