Authors : Neelu Verma

Volume/Issue : Volume 11 - 2026, Issue 9 - September

Google Scholar : https://tinyurl.com/28vwscjw

DOI : https://doi.org/10.38124/ijisrt/26sep1461

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Abstract : Large language models (LLMs) can produce fluent responses that remain unreliable when a question is rephrased, surrounded by irrelevant context, or intrinsically ambiguous. Most prompt-based hallucination mitigation methods optimize answer quality or average factual error, whereas deployment also requires a decision about whether an answer should be released at all. This paper presents Risk-Calibrated Prompting (RCP), an inference-time selective answering framework that combines semantic consistency across controlled prompt perturbations with model-verbalized confidence. RCP constructs a canonical prompt, a meaning-preserving paraphrase, and a controlled stress variant; obtains independent responses from the same model; estimates semantic agreement; combines agreement with verbalized confidence through a calibrated risk score; and releases the canonical answer only when the score crosses a validation-selected threshold. The framework requires neither parameter fine-tuning nor an external retrieval index. We define a paired evaluation protocol over TruthfulQA and the question-answering subset of HaluEval using open-weight instruction models, fixed perturbation templates, risk–coverage analysis, calibration metrics, robustness measures, cost accounting, and human audit. The protocol explicitly separates uncertainty signaling from factual truth: high agreement is treated as evidence of behavioral stability rather than proof of correctness. This formulation provides a reproducible basis for studying when an LLM should answer, when it should abstain, and how selective reliability changes under prompt perturbations.

Keywords : Large Language Models; Hallucination; Uncertainty Calibration; Selective Prediction; Abstention; Prompt Robustness; Semantic Consistency; Reliability.

References :

1. S. Lin, J. Hilton, and O. Evans, “TruthfulQA: Measuring How Models Mimic Human Falsehoods,” Proceedings of ACL, 2022, pp. 3214–3252. doi:10.18653/v1/2022.acl-long.229.
2. J. Li, X. Cheng, X. Zhao, J.-Y. Nie, and J.-R. Wen, “HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models,” Proceedings of EMNLP, 2023, pp. 6449–6464. doi:10.18653/v1/2023.emnlp-main.397.
3. S. Ji et al., “Survey of Hallucination in Natural Language Generation,” ACM Computing Surveys, vol. 55, no. 12, 2023.
4. P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” Advances in Neural Information Processing Systems, 2020.
5. S. Min et al., “FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation,” Proceedings of EMNLP, 2023, pp. 12076–12100.
6. P. Manakul, A. Liusie, and M. Gales, “SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models,” Proceedings of EMNLP, 2023, pp. 9004–9017. doi:10.18653/v1/2023.emnlp-main.557.
7. L. Kuhn, Y. Gal, and S. Farquhar, “Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation,” arXiv:2302.09664, 2023.
8. S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal, “Detecting Hallucinations in Large Language Models Using Semantic Entropy,” Nature, vol. 630, pp. 625–630, 2024. doi:10.1038/s41586-024-07421-0.
9. C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On Calibration of Modern Neural Networks,” Proceedings of ICML, PMLR 70, 2017, pp. 1321–1330.
10. Y. Geifman and R. El-Yaniv, “Selective Classification for Deep Neural Networks,” Advances in Neural Information Processing Systems, vol. 30, 2017.
11. X. Huang et al., “Uncertainty in Language Models: Assessment through Rank-Calibration,” arXiv:2404.03163, 2024.
12. J. P. Wahle, T. Ruas, Y. Xu, and B. Gipp, “Paraphrase Types Elicit Prompt Engineering Capabilities,” Proceedings of EMNLP, 2024, pp. 11004–11033. doi:10.18653/v1/2024.emnlp-main.617.
13. F. Shi et al., “Large Language Models Can Be Easily Distracted by Irrelevant Context,” Proceedings of ICML, PMLR 202, 2023, pp. 31210–31227.
14. S. Dhuliawala et al., “Chain-of-Verification Reduces Hallucination in Large Language Models,” arXiv:2309.11495, 2023.
15. J. Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” Advances in Neural Information Processing Systems, vol. 35, 2022.
16. X. Wang et al., “Self-Consistency Improves Chain of Thought Reasoning in Language Models,” Proceedings of ICLR, 2023.
17. C. Niu, Y. Wu, J. Zhu, S. Xu, K. Shum, R. Zhong, J. Song, and T. Zhang, “RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models,” Proceedings of ACL, 2024, pp. 10862–10878. doi:10.18653/v1/2024.acl-long.585.
18. J. Kadavath et al., “Language Models (Mostly) Know What They Know,” arXiv:2207.05221, 2022.
19. Y. Zhou et al., “Large Language Models Are Human-Level Prompt Engineers,” Proceedings of ICLR, 2023.
20. Y. Yao et al., “Tree of Thoughts: Deliberate Problem Solving with Large Language Models,” Advances in Neural Information Processing Systems, vol. 36, 2023.
21. N. Verma, A. De, and A. Mishra, “Bridging Language to Visuals: Towards Natural Language Query-to-Chart Image Retrieval,” International Journal of Multimedia Information Retrieval, vol. 13, no. 3, p. 32, 2024.
22. N. Verma and A. Mishra, “Retrieval-Augmented Chart-Grounded Fact Verification with Self-Consistent Vision–Language Reasoning,” International Journal of Data Science and Analytics, 2026.
23. N. Verma, “LST-ResNet: Optimization-Based Hybrid Network Architecture for an Effective Question and Answering System with Bar Chart Images,” International Journal of Scientific Research and Technology, vol. 3, no. 9, pp. 243–265, 2026.
24. N. Verma, “ASBO-GRU: Optimization-Enhanced Contextual and Aspect-Aware Neural Summarization,” International Journal of Innovative Research in Technology, vol. 13, no. 4, pp. 2548–2559, 2026.
25. N. Verma, “ChartSynth: Structured Summarization and Comparison of Multi-Chart Scientific Figures,” International Journal of Science and Research, vol. 15, no. 9, pp. 1230–1238, 2026. doi:10.21275/SR26919202852.
26. N. Verma, “AERIS-Chart: Evidence-Sufficient and Uncertainty-Calibrated Retrieval for Chart Claim Verification,” manuscript submitted, 2026.