Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models

Published

July 21, 2026

Author(s)

Anoop Singhal, Tahmid Siddiqui, Akib Ononto, Latifur Khan

Abstract

Large language models (LLMs) have seen widespread adoption across various domains, yet their reliability is frequently undermined by hallucinations - responses that are plausible-sounding but factually incorrect. In high-stakes domains, these errors can reduce trust and introduce real-world risk. To address this challenge, we present a parameter-efficient approach that uses soft prompts to mitigate hallucinated content and promote responsible abstention in generative question-answering (QA) tasks. Our method, called Responsible Contrastive Soft Prompting (RCSP), uses a composite loss to train soft prompts that balance three goals: suppressing hallucinatory content, encouraging abstention under uncertainty, and preserving or improving factual recall. To achieve these goals, we incorporate contrastive loss, curriculum learning, and KL regularization into our training mechanism. We evaluate our approach on five diverse generative QA datasets using an LLM-as-a-Judge framework. Experimental results on the Gemma 3 (12B) and Llama 3.1 (8B) backbones demonstrate that RCSP effectively balances factual recall with hallucination suppression and abstention, yielding a generally superior F-score over standard reasoning and instruction-based prompting baselines. Notably, these improvements are achieved by training only a fraction of the parameters required by other tuning techniques. Our results demonstrate that soft prompts provide a modular and computationally efficient path toward improving LLM reliability.

Proceedings Title

Data and Applications Security and Privacy XL (DBSec 2026)

Citation

<i>Data and Applications Security and Privacy XL (DBSec 2026)</i> (IFIP Conference on Data and Application Security DBSEC 2026, Arlington, VA, US, 28-JUL-26 to 30-JUL-26)

Volume

16576

Conference Dates

2026-07-28T00:00:00

Conference Location

Arlington, VA, US

Conference Title

IFIP Conference on Data and Application Security DBSEC 2026

Pub Type

Conferences

Download Paper

https://doi.org/10.1007/978-3-032-33260-8_14

Local Download

Keywords

large language models, hallucinations, soft prompts, prompt tuning, parameter-efficient tuning

Citation

Singhal, A. , Siddiqui, T. , Ononto, A. and Khan, L. (2026), Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models, Data and Applications Security and Privacy XL (DBSec 2026), Arlington, VA, US, [online], https://doi.org/10.1007/978-3-032-33260-8_14, https://nps-ext.nist.gov/nps/nps_public_api/api/Publication/file/961839 (Accessed November 13, 2026)

Additional citation formats