Beyond the Answer: Faithful and Counterfactual Explanations for LLM-Powered Chatbots

Large Language Models (LLMs) have significantly advanced the development of intelligent conversational systems. However, their limited transparency and potential to generate unsupported information raise concerns about the reliability of their responses, particularly in security-critical applications. Retrieval-Augmented Generation (RAG) addresses some of these limitations by incorporating external knowledge into the response generation process. Nevertheless, providing citations to retrieved documents does not necessarily demonstrate that these documents actually influenced the generated answer [1], [2]. Recent research on context attribution and counterfactual reasoning has highlighted the importance of identifying influential evidence and understanding how changes in the retrieved context affect model behavior [3], [4]. This thesis aims to develop an explainable RAG-based chatbot capable of providing not only evidence-supported answers but also faithful explanations of how retrieved information contributes to its responses. The proposed framework will investigate the relationship between retrieved evidence, generated claims, and the underlying behavior of the language model, with a particular focus on cybersecurity question-answering applications.

The student will design and implement a novel explanation framework that combines evidence attribution, counterfactual analysis, and explanation faithfulness verification. The proposed approach will identify influential passages within retrieved documents, generate counterfactual explanations by systematically modifying or removing selected evidence, and evaluate whether the resulting changes in the model's responses are consistent with the explanations provided. To improve computational efficiency, the student will investigate evidence-selection strategies that reduce the number of model evaluations required for explanation generation. The framework will be integrated into an interactive cybersecurity chatbot using a domain-specific knowledge base, such as MITRE ATT&CK. Its effectiveness will be evaluated against existing attribution methods and conventional RAG systems using established benchmarks, including ALCE and RAGBench [5], [6]. Experimental evaluation will consider answer correctness, citation quality, attribution accuracy, explanation faithfulness, robustness to irrelevant or conflicting evidence, and computational overhead. The expected outcome is a reproducible explainable chatbot framework and an empirical assessment of the relationship between evidence attribution, counterfactual explanations, and response reliability.

References 

[1] J. Wallat, M. Heuss, M. de Rijke, and A. Anand, “Correctness is not faithfulness in retrieval augmented generation attributions,” in Proc. 2025 Int. ACM SIGIR Conf. Innovative Concepts and Theories in Information Retrieval (ICTIR), 2025, pp. 22–32, doi: 10.1145/3731120.3744592.

[2] T. Gao, H. Yen, J. Yu, and D. Chen, “Enabling large language models to generate text with citations,” in Proc. 2023 Conf. Empirical Methods in Natural Language Processing (EMNLP), 2023, pp. 6465–6488, doi: 10.18653/v1/2023.emnlp-main.398.

[3] B. Cohen-Wang, H. Shah, K. Georgiev, and A. Mądry, “ContextCite: Attributing model generation to context,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 37, 2024, doi: 10.52202/079017-3035.

[4] H. Qin, C. Wei, Y. Chen, and Y. Wang, “Counterfactual reasoning for retrieval-augmented generation,” in Proc. Int. Conf. Learning Representations (ICLR), 2026. [Online]. Available: ICLR 2026 Paper

[5] R. Friel, M. Belyi, and A. Sanyal, “RAGBench: Explainable benchmark for retrieval-augmented generation systems,” arXiv preprint arXiv:2407.11005, 2024. [Online]. Available: arxiv.org/abs/2407.11005

[6] S. Es, J. James, L. Espinosa Anke, and S. Schockaert, “RAGAs: Automated evaluation of retrieval augmented generation,” in Proc. 18th Conf. European Chapter of the Association for Computational Linguistics: System Demonstrations (EACL), 2024, pp. 150–158, doi: 10.18653/v1/2024.eacl-demo.16.