BreastCancerDiagNet - Transformer-Based Clinical Question Generation for Automated History Taking
Received: 20 September 2025 | Revised: 11 October 2025 and 23 October 2025 | Accepted: 24 October 2025 | Online: 7 February 2026
Corresponding author: Maleeha Fathima
Abstract
History-taking is a highly significant procedure in clinical decision-making that remains time-consuming, inconsistent, and can lead to omissions. This study presents BreastCancerDiagNet, a complex transformer-powered system to automate medical history-taking and aid in diagnosis. This model uses structured patient demographics and unstructured clinical symptoms with hybridized ClinicalBERT embeddings, BiLSTM sequence modeling, and self-attention mechanisms. These capabilities are integrated in an encoder-decoder architecture with rotary position embeddings and FlashAttention to enable long-sequence processing. A Reinforcement Learning with Human Feedback (RLHF) strategy is used to refine the question generation strategy to reflect contextual reference to clinical practice. The proposed system was trained and tested using a breast cancer dataset of curated demographic data, symptom data, comorbidities, lifestyle indicators, and physician-curated ground truth questionnaires. The results show that BreastCancerDiagNet achieved a BLEU-4 score of 0.42, a ROUGE-L score of 0.56, and a BERT-F1 score of 0.88, which are higher than the Seq2Seq and Vanilla Transformer baselines. Qualitative analysis confirmed the relevance of the questions generated in clinical practice, covering lump presence, pain, discharge, family history, and drug use. The findings demonstrate the possibility of using BreastCancerDiagNet to save time in consultations, reduce the number of diagnostic errors, and serve as a future-generation Clinical Decision Support System (CDSS) that can be scaled and interpreted.
Keywords:
medical question generation, transformer, clinicalBERT, reinforcement learning with human feedback, breast cancer diagnosis, clinical decision supportDownloads
References
K. Nassiri and M. A. Akhloufi, ''Recent Advances in Large Language Models for Healthcare,'' BioMedInformatics, vol. 4, no. 2, pp. 1097–1143, Apr. 2024. DOI: https://doi.org/10.3390/biomedinformatics4020062
S. Chatterjee, A. Fruhling, K. Kotiadis, and D. Gartner, ''Towards new frontiers of healthcare systems research using artificial intelligence and generative AI,'' Health Systems, vol. 13, no. 4, pp. 263–273, Oct. 2024. DOI: https://doi.org/10.1080/20476965.2024.2402128
H. Zhou et al., ''A Survey of Large Language Models in Medicine: Progress, Application, and Challenge.'' arXiv, July 23, 2024.
H. Yadav, P. Yadav, N. Yadav, and P. Chaudhary, ''AI in Healthcare: A Survey on Medical Question Answering System,'' South Eastern European Journal of Public Health, pp. 1287–1298, Dec. 2024. DOI: https://doi.org/10.70135/seejph.vi.2683
G. Kell et al., ''Question answering systems for health professionals at the point of care—a systematic review,'' Journal of the American Medical Informatics Association, vol. 31, no. 4, pp. 1009–1024, Apr. 2024. DOI: https://doi.org/10.1093/jamia/ocae015
M. Sarrouti and S. O. El Alaoui, ''SemBioNLQA: A semantic biomedical question answering system for retrieving exact and ideal answers to natural language questions,'' Artificial Intelligence in Medicine, vol. 102, Jan. 2020, Art. no. 101767. DOI: https://doi.org/10.1016/j.artmed.2019.101767
H. Faris, M. Habib, M. Faris, A. Alomari, P. A. Castillo, and M. Alomari, ''Classification of Arabic healthcare questions based on word embeddings learned from massive consultations: a deep learning approach,'' Journal of Ambient Intelligence and Humanized Computing, vol. 13, no. 4, pp. 1811–1827, Apr. 2022. DOI: https://doi.org/10.1007/s12652-021-02948-w
S. Canchila, C. Meneses-Eraso, J. Casanoves-Boix, P. Cortés-Pellicer, and F. Castelló-Sirvent, ''Natural language processing: An overview of models, transformers and applied practices,'' Computer Science and Information Systems, vol. 21, no. 3, pp. 1097–1145, 2024. DOI: https://doi.org/10.2298/CSIS230217031C
P. Rouzrokh et al., ''A Current Review of Generative AI in Medicine: Core Concepts, Applications, and Current Limitations,'' Current Reviews in Musculoskeletal Medicine, vol. 18, no. 7, pp. 246–266, Apr. 2025. DOI: https://doi.org/10.1007/s12178-025-09961-y
K. Singhal et al., ''Toward expert-level medical question answering with large language models,'' Nature Medicine, vol. 31, no. 3, pp. 943–950, Mar. 2025. DOI: https://doi.org/10.1038/s41591-024-03423-7
M. Cascella, F. Semeraro, J. Montomoli, V. Bellini, O. Piazza, and E. Bignami, ''The Breakthrough of Large Language Models Release for Medical Applications: 1-Year Timeline and Perspectives,'' Journal of Medical Systems, vol. 48, no. 1, Feb. 2024, Art. no. 22. DOI: https://doi.org/10.1007/s10916-024-02045-3
J. Lee et al., "BioBERT: a pre-trained biomedical language representation model for biomedical text mining," Bioinformatics, vol. 36, no. 4, pp. 1234–1240, Feb. 2020. DOI: https://doi.org/10.1093/bioinformatics/btz682
A. Chaddad, J. Peng, J. Xu, and A. Bouridane, "Survey of Explainable AI Techniques in Healthcare," Sensors, vol. 23, no. 2, Jan. 2023, Art. no. 634. DOI: https://doi.org/10.3390/s23020634
D. Wang and S. Zhang, "Large hlanguage models in medical and healthcare fields: applications, advances, and challenges," Artificial Intelligence Review, vol. 57, no. 11, Sept. 2024, Art. no. 299. DOI: https://doi.org/10.1007/s10462-024-10921-0
A. Nentidis, A. Krithara, K. Bougiatiotis, G. Paliouras, and I. Kakadiaris, "Results of the sixth edition of the BioASQ Challenge," in Proceedings of the 6th BioASQ Workshop A challenge on large-scale biomedical semantic indexing and question answering, Brussels, Belgium, 2018, pp. 1–10. DOI: https://doi.org/10.18653/v1/W18-5301
C. Y. Lin, "ROUGE: A Package for Automatic Evaluation of Summaries," in Text Summarization Branches Out, Barcelona, Spain, Apr. 2004, pp. 74–81.
K. Papineni, S. Roukos, T. Ward, and W. J. Zhu, "BLEU: a method for automatic evaluation of machine translation," in Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL ’02, Philadelphia, PA, USA, 2001, Art. no. 311. DOI: https://doi.org/10.3115/1073083.1073135
M. Reid, M. French, S. Andreopoulos, C. Wong, and N. Kee, "AI-generated multiple-choice questions in health science education: Stakeholder perspectives and implementation considerations," Current Research in Physiology, vol. 8, Jan. 2025, Art. no. 100160. DOI: https://doi.org/10.1016/j.crphys.2025.100160
S. Shen et al., "On the Generation of Medical Question-Answer Pairs," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, pp. 8822–8829, Apr. 2020. DOI: https://doi.org/10.1609/aaai.v34i05.6410
Y. Cheng et al., "Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting," in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, 2021, pp. 5968–5978. DOI: https://doi.org/10.18653/v1/2021.acl-long.465
T. Searle, Z. Ibrahim, J. Teo, and R. J. B. Dobson, "Discharge summary hospital course summarisation of in patient Electronic Health Record text with clinical concept guided deep pre-trained Transformer models," Journal of Biomedical Informatics, vol. 141, May 2023, Art. no. 104358. DOI: https://doi.org/10.1016/j.jbi.2023.104358
P. Chakraborty, T. Chandrapragasam, A. Arunachalam, and S. Rafiammal, ''Artificial Intelligence-based Oral Cancer Screening System using Smartphones,'' Engineering, Technology & Applied Science Research, vol. 13, no. 6, pp. 12054–12057, Dec. 2023. DOI: https://doi.org/10.48084/etasr.6364
T. Siddiqui, M. Latif, M. U. Farooq, M. A. Baig, and Y. S. Hassan, ''Chronic Obstructive Pulmonary Disease Diagnosis with Bagging Ensemble Learning and ANN Classifiers,'' Engineering, Technology & Applied Science Research, vol. 14, no. 3, pp. 14741–14746, June 2024. DOI: https://doi.org/10.48084/etasr.7106
Downloads
How to Cite
License
Copyright (c) 2025 Maleeha Fathima, Moulana Mohammed
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
