LLM – System prompt – Prevent contact hallucination with a registry

When a conversational assistant answers “How do I contact service X?”, it sometimes invents a plausible but wrong phone number or email. The cause is a training bias: the model has seen thousands of phone numbers and is very good at producing one that “looks real”. RAG alone does not solve this, because if the contact is missing from the retrieved passages, the model fills the gap. The reliable fix is to inject a registry of verified contacts into the system prompt with a strict instruction. This pattern drastically reduces LLM hallucination on structured data.

Define the registry as the source of truth

The registry is a contact list maintained outside the RAG content. Store it as CSV or JSON, parse it at app startup, format it for the prompt at each request. The CSV format lets a non-technical admin update contacts via the UI without a deploy.

interface ServiceEntry {
  name: string;
  phone: string;
  email: string;
  contactUrl: string;
}

function parseRegistry(csv: string): ServiceEntry[] {
  return csv
    .split('\n')
    .filter((line) => line && !line.startsWith('#'))
    .slice(1)
    .map((row) => {
      const [name, phone, email, contactUrl] = row.split(';').map((f) => f.trim());
      return { name, phone, email, contactUrl };
    });
}

Inject the registry into the system prompt

The format sent to the LLM must stay compact and explicit. A negative instruction in uppercase, placed before the list, strongly reduces LLM hallucination in internal A/B tests.

function buildSystemPrompt(registry: ServiceEntry[]): string {
  const lines = registry
    .map((s) => `- ${s.name}: phone ${s.phone}, email ${s.email}, ${s.contactUrl}`)
    .join('\n');

  return `You are the official assistant for the organization.
STRICT RULE: for any contact question, answer ONLY
with the data below. Never invent a phone number or email.

CONTACT REGISTRY:
${lines}`;
}

LLM hallucination: the registry complements RAG

RAG is still useful for free-form text such as procedures, FAQs, or articles. The registry covers structured data where LLM hallucination is unacceptable: contacts, opening hours, prices, identifiers. This separation keeps hallucinations rare even with smaller models, and the registry stays easy to audit because it is versioned independently from RAG chunks.