Contextual Query Expansion: Transforming Question Answering with an MLM-Based Approach Using A Transformer Model
Received: 4 October 2025 | Revised: 12 November 2025 and 5 December 2025 | Accepted: 7 December 2025 | Online: 16 December 2025
Corresponding author: Ahmad Shaf
Abstract
The purpose of Question Answering (QA) frameworks is to provide precise answers to Natural Language Queries (NLQ) by extracting relevant information from large document collections. A key challenge is that users often struggle to formulate optimal queries, a limitation that prior methods have often failed to address by neglecting the valuable role of contextual information. To overcome this, this study introduces an MLM-based approach that takes advantage of contextual cues from document collection. By harnessing the context extraction capabilities of transformer models (such as BERT), expanded queries are automatically generated to better capture user intent. The proposed technique significantly improves the effectiveness of QA. The proposed MLM-generated expanded queries outperform query expansion methods based on WordNet, validating the integration of context clues as a promising development to refine the accuracy and relevance of QA responses.
Keywords:
questions, answers, query expansion, natural language processing, information retrieval, MLM, BERT, contextualize query expansionDownloads
References
W. Zheng, H. Cheng, J. X. Yu, L. Zou, and K. Zhao, "Interactive natural language question answering over knowledge graphs," Information Sciences, vol. 481, pp. 141–159, May 2019. DOI: https://doi.org/10.1016/j.ins.2018.12.032
O. Kolomiyets and M. F. Moens, "A survey on question answering technology from an information retrieval perspective," Information Sciences, vol. 181, no. 24, pp. 5412–5434, Dec. 2011. DOI: https://doi.org/10.1016/j.ins.2011.07.047
T. Hao, W. Xie, Q. Wu, H. Weng, and Y. Qu, "Leveraging question target word features through semantic relation expansion for answer type classification," Knowledge-Based Systems, vol. 133, pp. 43–52, Oct. 2017. DOI: https://doi.org/10.1016/j.knosys.2017.06.030
H. Toba, Z. Y. Ming, M. Adriani, and T. S. Chua, "Discovering high quality answers in community question answering archives using a hierarchy of classifiers," Information Sciences, vol. 261, pp. 101–115, Mar. 2014. DOI: https://doi.org/10.1016/j.ins.2013.10.030
B. Cabaleiro, A. Peñas, and S. Manandhar, "Grounding proposition stores for question answering over linked data," Knowledge-Based Systems, vol. 128, pp. 34–42, July 2017. DOI: https://doi.org/10.1016/j.knosys.2017.04.016
H. J. Oh, S. H. Myaeng, and M. G. Jang, "Semantic passage segmentation based on sentence topics for question answering," Information Sciences, vol. 177, no. 18, pp. 3696–3717, Sept. 2007. DOI: https://doi.org/10.1016/j.ins.2007.02.038
Z. Yan, N. Duan, J. Bao, P. Chen, M. Zhou, and Z. Li, "Response selection from unstructured documents for human-computer conversation systems," Knowledge-Based Systems, vol. 142, pp. 149–159, Feb. 2018. DOI: https://doi.org/10.1016/j.knosys.2017.11.033
W. Wei et al., "Exploring heterogeneous features for query-focused summarization of categorized community answers," Information Sciences, vol. 330, pp. 403–423, Feb. 2016. DOI: https://doi.org/10.1016/j.ins.2015.10.024
A. G. Tapeh and M. Rahgozar, "A knowledge-based question answering system for B2C eCommerce," Knowledge-Based Systems, vol. 21, no. 8, pp. 946–950, Dec. 2008. DOI: https://doi.org/10.1016/j.knosys.2008.04.005
F. Wang, W. Wu, Z. Li, and M. Zhou, "Named entity disambiguation for questions in community question answering," Knowledge-Based Systems, vol. 126, pp. 68–77, June 2017. DOI: https://doi.org/10.1016/j.knosys.2017.03.017
S. J. Yen, Y. C. Wu, J. C. Yang, Y. S. Lee, C. J. Lee, and J. J. Liu, "A support vector machine-based context-ranking model for question answering," Information Sciences, vol. 224, pp. 77–87, Mar. 2013. DOI: https://doi.org/10.1016/j.ins.2012.10.014
A. Rodrigo and A. Peñas, "A study about the future evaluation of Question-Answering systems," Knowledge-Based Systems, vol. 137, pp. 83–93, Dec. 2017. DOI: https://doi.org/10.1016/j.knosys.2017.09.015
B. Selvaretnam and M. Belkhatir, "Natural language technology and query expansion: issues, state-of-the-art and perspectives," Journal of Intelligent Information Systems, vol. 38, no. 3, pp. 709–740, June 2012. DOI: https://doi.org/10.1007/s10844-011-0174-3
M. Habibi, P. Mahdabi, and A. Popescu-Belis, "Question answering in conversations: Query refinement using contextual and semantic information," Data & Knowledge Engineering, vol. 106, pp. 38–51, Nov. 2016. DOI: https://doi.org/10.1016/j.datak.2016.06.003
S. Momtazi and D. Klakow, "Bridging the vocabulary gap between questions and answer sentences," Information Processing & Management, vol. 51, no. 5, pp. 595–615, Sept. 2015. DOI: https://doi.org/10.1016/j.ipm.2015.04.005
Y. Gupta and A. Saini, "A novel Fuzzy-PSO term weighting automatic query expansion approach using combined semantic filtering," Knowledge-Based Systems, vol. 136, pp. 97–120, Nov. 2017. DOI: https://doi.org/10.1016/j.knosys.2017.09.004
F. Diaz, B. Mitra, and N. Craswell, "Query Expansion with Locally-Trained Word Embeddings," presented at the Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, June 2016. DOI: https://doi.org/10.18653/v1/P16-1035
A. Figueroa, "Automatically generating effective search queries directly from community question-answering questions for finding related questions," Expert Systems with Applications, vol. 77, pp. 11–19, July 2017. DOI: https://doi.org/10.1016/j.eswa.2017.01.041
F. C. Fernández-Reyes, J. Hermosillo-Valadez, and M. Montes-y-Gómez, "A Prospect-Guided global query expansion strategy using word embeddings," Information Processing & Management, vol. 54, no. 1, pp. 1–13, Jan. 2018. DOI: https://doi.org/10.1016/j.ipm.2017.09.001
P. Azevedo, B. Leite, H. L. Cardoso, D. C. Silva, and L. P. Reis, "Exploring NLP and Information Extraction to Jointly Address Question Generation and Answering," in Artificial Intelligence Applications and Innovations, 2020, pp. 396–407. DOI: https://doi.org/10.1007/978-3-030-49186-4_33
K. Karpagam and A. Saradha, "A framework for intelligent question answering system using semantic context-specific document clustering and Wordnet," Sādhanā, vol. 44, no. 3, Feb. 2019, Art. no. 62. DOI: https://doi.org/10.1007/s12046-018-1022-8
V. Dibia, "NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online, July 2020, pp. 15–22. DOI: https://doi.org/10.18653/v1/2020.emnlp-demos.3
M. Esposito, E. Damiano, A. Minutolo, G. De Pietro, and H. Fujita, "Hybrid query expansion using lexical resources and word embeddings for sentence retrieval in question answering," Information Sciences, vol. 514, pp. 88–105, Apr. 2020. DOI: https://doi.org/10.1016/j.ins.2019.12.002
E. Damiano, A. Minutolo, S. Silvestri, and M. Esposito, "Query Expansion Based on WordNet and Word2vec for Italian Question Answering Systems," in Advances on P2P, Parallel, Grid, Cloud and Internet Computing, 2018, pp. 301–313. DOI: https://doi.org/10.1007/978-3-319-69835-9_29
I. Lahbari, S. E. Alaoui, and K. Zidani, "Toward a New Arabic Question Answering System," The International Arab Journal of Information Technology, vol. 15, no. 3A, pp. 610–619, 2018.
W. Bakari, P. Bellot, and M. Neji, "A logical representation of Arabic questions toward automatic passage extraction from the Web," International Journal of Speech Technology, vol. 20, no. 2, pp. 339–353, June 2017. DOI: https://doi.org/10.1007/s10772-017-9411-7
H. K. Azad and A. Deepak, "A new approach for query expansion using Wikipedia and WordNet," Information Sciences, vol. 492, pp. 147–163, Aug. 2019. DOI: https://doi.org/10.1016/j.ins.2019.04.019
S. Kandasamy and A. K. Cherukuri, "Query expansion using named entity disambiguation for a question-answering system," Concurrency and Computation: Practice and Experience, vol. 32, no. 4, 2020, Art. no. e5119. DOI: https://doi.org/10.1002/cpe.5119
D. Parapar, A. Barreiro, and D. E. Losada, "Query expansion using wordnet with a logical model of information retrieval," in IADIS AC, 2005, pp. 487–494.
R. Chauhan, R. Goudar, R. Rathore, P. Singh, and S. Rao, "Ontology Based Automatic Query Expansion for Semantic Information Retrieval in Sports Domain," in Eco-friendly Computing and Communication Systems, 2012, pp. 422–433. DOI: https://doi.org/10.1007/978-3-642-32112-2_49
P. Sharma and N. Joshi, "Knowledge-Based Method for Word Sense Disambiguation by Using Hindi WordNet," Engineering, Technology & Applied Science Research, vol. 9, no. 2, pp. 3985–3989, Apr. 2019. DOI: https://doi.org/10.48084/etasr.2596
D. Chopra, N. Joshi, and I. Mathur, "Improving Translation Quality By Using Ensemble Approach," Engineering, Technology & Applied Science Research, vol. 8, no. 6, pp. 3512–3514, Dec. 2018. DOI: https://doi.org/10.48084/etasr.2269
B. Nethravathi, G. Amitha, A. Saruka, T. P. Bharath, and S. Suyagya, "Structuring Natural Language to Query Language: A Review," Engineering, Technology & Applied Science Research, vol. 10, no. 6, pp. 6521–6525, Dec. 2020. DOI: https://doi.org/10.48084/etasr.3873
P. Rajpurkar, R. Jia, and P. Liang, "Know What You Don't Know: Unanswerable Questions for SQuAD." arXiv, June 11, 2018. DOI: https://doi.org/10.18653/v1/P18-2124
Downloads
How to Cite
License
Copyright (c) 2025 Muhammad Manzoor Faisal, Javed Ferzund, Ahmad Shaf, Afnan Aldhahri
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
