Evaluating Identifier Readability Using CodeBERT Embeddings and Self-Attention Bi-LSTM with Explainable Modeling
Corresponding author: Bharat Babaso Mane
Abstract
Identifier names are natural language representations in source code that play a significant role in program understanding. Studies on the quality of identifier names generally focus on their role in program knowledge. Identifier names and naming conventions are important for programming understanding. Previous studies have shown a connection between software quality and the quality of identifier names. Recently, Deep Learning (DL) has been used to develop highly efficient models for identifying intelligibility challenges. DL models, such as transformer-based frameworks and attention mechanisms, can acquire contextual and sequential relations between identified tokens. This study proposes an Identifier Readability Analysis Framework using an Explainable Attention-Based Deep Learning (IRAF-XADL) approach, with the primary intention of assessing the quality of identifier names in Python and C++ source code. In the initial stage, a syntax-aware identifier preprocessing pipeline based on language-specific abstract syntax tree parsing is applied to extract identifiers and perform lexical normalization and semantic cleaning. From the normalized identifiers, this study computes ten linguistically and cognitively grounded readability parameters. Semantic and contextual representations are attained using CodeBERT embeddings, which are then processed by a self-attention-based bidirectional long short-term memory model to learn sequential and contextual dependencies. Furthermore, the model is optimized using the AdamW optimizer, enabling improved convergence and overall performance. In the last stage, SHAP-based explainability is integrated for interpreting the contribution of identifier tokens and features to readability predictions. The IRAF-XADL method was experimentally examined on the benchmark Code Snippets: Insights and Readability Dataset, and the results prove the improved performance over the existing approaches in terms of diverse metrics.
Keywords:
identifier readability, code snippets, CodeBERT embeddings, model optimization, Explainable Artificial Intelligence (XAI), deep learningReferences
S. Fakhoury, Y. Ma, V. Arnaoudova, and O. Adesope, "The effect of poor source code lexicon and readability on developers’ cognitive load," in Proceedings of the 26th Conference on Program Comprehension, Feb. 2018, pp. 286–296.
D. Oliveira, R. Santos, B. de Oliveira, M. Monperrus, F. Castor, and F. Madeiral, "Understanding Code Understandability Improvements in Code Reviews," IEEE Transactions on Software Engineering, vol. 51, no. 1, pp. 14–37, Jan. 2025.
H. Mestiri, I. Barraj, and M. Machhout, "AES High-Level SystemC Modeling using Aspect Oriented Programming Approach," Engineering, Technology & Applied Science Research, vol. 11, no. 1, pp. 6719–6723, Feb. 2021.
S. Butler, M. Wermelinger, Yijun Yu, and H. Sharp, "Exploring the Influence of Identifier Names on Code Quality: An Empirical Study," in 2010 14th European Conference on Software Maintenance and Reengineering, Mar. 2010, pp. 156–165.
E. D. Berger, C. Hollenbeck, P. Maj, O. Vitek, and J. Vitek, "On the Impact of Programming Languages on Code Quality: A Reproduction Study," ACM Transactions on Programming Languages and Systems (TOPLAS), vol. 41, no. 4, July 2019, Art. no. 21.
R. A. Al-Msie’deen, "Tag Clouds for Object-Oriented Source Code Visualization," Engineering, Technology & Applied Science Research, vol. 9, no. 3, pp. 4243–4248, June 2019.
W. Zeng, Y. Chai, H. Zhou, F. Meng, J. Zhou, and X. Gu, "Readability-Robust Code Summarization via Meta Curriculum Learning." arXiv, 2026.
A. Midolo, E. Tramontana, and M. Di Penta, "From Human to Machine Refactoring: Assessing GPT-4’s Impact on Python Class Quality and Readability." arXiv, 2026.
S. Tokumoto, S. Kusumoto, and R. Imai, "Development and Evaluation of a Deep Learning-Based Model for Source Code Quality Classification Using Industrial Data," Journal of Software Engineering Practice, vol. 6, no. 1, pp. 1–19, June 2025.
A. Verma, R. Saha, G. Kumar, A. Brighente, M. Conti, and T. H. Kim, "Exploring the Landscape of Programming Language Identification With Machine Learning Approaches," IEEE Access, vol. 13, pp. 23556–23579, 2025.
M. Biagiola, G. Ghislotti, and P. Tonella, "Improving the Readability of Automatically Generated Tests Using Large Language Models," in 2025 IEEE Conference on Software Testing, Verification and Validation (ICST), Mar. 2025, pp. 162–173.
H. Lu et al., "Malsight: Exploring Malicious Source Code and Benign Pseudocode for Iterative Binary Malware Summarization," IEEE Transactions on Information Forensics and Security, vol. 20, pp. 6733–6747, 2025.
Z. Feng et al., "CodeBERT: A Pre-Trained Model for Programming and Natural Languages," in Findings of the Association for Computational Linguistics: EMNLP 2020, Aug. 2020, pp. 1536–1547.
A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures," Neural Networks, vol. 18, no. 5, pp. 602–610, July 2005.
I. Loshchilov and F. Hutter, "Decoupled Weight Decay Regularization." arXiv, 2017.
P. Maheshwari, "Code Snippets: Insights and Readability." Kaggle, [Online]. Available: https://www.kaggle.com/datasets/paakhim10/code-snippets-insights-and-readability.
Q. Mi, J. Keung, Y. Xiao, S. Mensah, and X. Mei, "An Inception Architecture-Based Model for Improving Code Readability Classification," in Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018, Mar. 2018, pp. 139–144.
B. Susanto, R. Ferdiana, and T. B. Adji, "Predicting Multiclass Java Code Readability: A Comparative Study of Machine Learning Algorithms," International Journal of Advanced Computer Science and Applications, vol. 16, no. 4, 2025.
Downloads
How to Cite
License
Copyright (c) 2026 Bharat Babaso Mane, Rathnakar Achary
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
