Detecting Contextual Hallucinations in Large Language Models with Frequency-Aware Attention

This paper introduces a frequency-aware perspective on attention: we model attention distributions during generation as discrete signals and extract their high-frequency components, which reflect rapid local changes in attention. We show that hallucinated tokens are associated with high-frequency attention energy, indicating fragmented and unstable grounding, and build a lightweight hallucination detector on these features that outperforms verification-based, internal-representation-based, and attention-based methods on RAGTruth and HalluRAG across models and tasks.

Recommended citation: S Qi, Y Chen, R Zhao, Q Zhu, Z Hu, W Liu, Y He, Z Yuan, L Gui. (2026). "Detecting Contextual Hallucinations in Large Language Models with Frequency-Aware Attention." Forty-third International Conference on Machine Learning (ICML 2026).
Download Paper