Language models hallucinate when they generate facts not grounded in your actual data. Stop this in n8n by implementing retrieval-augmented generation (RAG) with vector stores, adding explicit grounding instructions in the system prompt, using a verification Code node to cross-check LLM outputs against source data, and setting temperature to 0 for factual queries. These techniques ensure your chatbot only states what your data supports.
Why AI Chatbots Hallucinate and How to Ground Them in n8n
Hallucination happens when a language model generates plausible-sounding but factually incorrect information. In a customer-facing chatbot, this can mean inventing product features, quoting wrong prices, or fabricating policies. The root cause is that LLMs are trained to predict likely next tokens, not to verify truth. The solution is grounding: restricting the model to information you explicitly provide. In n8n, you achieve this through RAG (retrieval-augmented generation) using vector stores, strict system prompts that forbid speculation, and post-generation verification nodes that cross-check outputs against your actual data.
Prerequisites
- A running n8n instance (v1.30 or later)
- An LLM credential (OpenAI, Claude, or Gemini)
- A vector store (Pinecone, Qdrant, Supabase Vector, or PGVector) with your knowledge base indexed
- Basic understanding of embeddings and similarity search concepts
Step-by-step guide
Set up RAG with a vector store to provide grounding context
Set up RAG with a vector store to provide grounding context
The most effective anti-hallucination technique is giving the LLM only the information it needs to answer, retrieved from your own data. In n8n, use a Vector Store node (Pinecone, Qdrant, Supabase, or PGVector) connected to the AI Agent node's retriever input. The vector store converts the user's question into an embedding, finds the most similar documents, and passes them to the LLM as context. The LLM can only reference what the retriever provides.
1// AI Agent node configuration:2// Model: Claude 3.5 Sonnet (or GPT-4o)3// Retriever: Supabase Vector Store4// - Table: documents5// - Embedding column: embedding6// - Content column: content7// - Metadata column: metadata8// - Top K: 5 (retrieve 5 most relevant chunks)9// - Similarity threshold: 0.7Expected result: The AI Agent receives relevant knowledge base chunks as context for every user question
Write an anti-hallucination system prompt
Write an anti-hallucination system prompt
Your system prompt must explicitly instruct the model to only use information from the provided context and to admit when it does not know something. Use strong, direct language and include specific refusal phrases the model should use when the context does not contain the answer. This is your primary behavioral control against fabrication.
1const systemPrompt = `You are a TechCorp product assistant. You answer user questions using ONLY the information provided in the CONTEXT section below.23<grounding_rules>41. ONLY use facts from the CONTEXT section to answer questions.52. NEVER invent, assume, or extrapolate information not in the CONTEXT.63. If the CONTEXT does not contain the answer, say exactly: "I don't have that information in my knowledge base. Let me connect you with our support team at support@techcorp.com."74. NEVER guess prices, dates, feature availability, or compatibility.85. When quoting data from CONTEXT, cite it naturally: "According to our documentation..."96. If the user asks about something partially covered, answer only the covered parts and state what you don't have information about.10</grounding_rules>1112<output_rules>13- Prefix uncertain statements with "Based on available information..."14- Never say "I think" or "probably" — either you know it from CONTEXT or you don't15- If asked to compare with competitors, say: "I can only provide information about TechCorp products."16</output_rules>`;Expected result: Claude consistently refuses to answer questions not covered by the retrieved context
Set model temperature to 0 for factual responses
Set model temperature to 0 for factual responses
Temperature controls how 'creative' the model's output is. For a factual chatbot, set temperature to 0 (or as close as possible). This makes the model deterministic, always choosing the most likely (and usually most accurate) next token. In the Claude or OpenAI node, find the Temperature parameter and set it to 0. If you need some variation in phrasing, 0.1 is a safe maximum.
1// Claude node settings:2// Temperature: 03// Top P: 1 (default)4// Max Tokens: 10245//6// OpenAI node settings:7// Temperature: 08// Top P: 1 (default)9// Max Tokens: 1024Expected result: Model outputs are deterministic and factual rather than creative or speculative
Add a post-generation verification Code node
Add a post-generation verification Code node
Even with RAG and strict prompts, the model may occasionally rephrase context inaccurately. Add a Code node after the LLM that cross-checks the response against the original retrieved context. The node looks for specific claims (numbers, dates, feature names) and verifies they appear in the source data. If a claim cannot be verified, the node flags it.
1const llmResponse = $input.first().json.output || $input.first().json.text || '';2const retrievedContext = $('Vector Store').first()?.json?.documents || [];34// Combine all retrieved document content5const contextText = retrievedContext.map(d => d.pageContent || d.content || '').join(' ').toLowerCase();6const responseLower = llmResponse.toLowerCase();78// Check for number claims in the response9const numberPattern = /\$[\d,.]+|\d+%|\d+ (days|hours|minutes|gb|tb|mb|users|seats)/gi;10const claims = responseLower.match(numberPattern) || [];1112const unverified = [];13for (const claim of claims) {14 if (!contextText.includes(claim.trim())) {15 unverified.push(claim);16 }17}1819const isVerified = unverified.length === 0;2021return [{22 json: {23 response: llmResponse,24 isVerified,25 unverifiedClaims: unverified,26 claimsChecked: claims.length27 }28}];Expected result: Responses with unverifiable numeric claims are flagged for review or re-generation
Handle unverified responses with a fallback
Handle unverified responses with a fallback
Add an IF node after the verification Code node. If isVerified is false and the number of unverified claims exceeds a threshold (e.g., 2), route to a fallback path. The fallback can either re-prompt the LLM with a stricter instruction ('Respond using ONLY the following exact data: ...'), or return a safe default response asking the user to contact support for precise details.
1// IF node condition:2// {{ $json.isVerified === false && $json.unverifiedClaims.length >= 2 }}3//4// Fallback response:5// "I want to make sure I give you accurate information. For specific pricing and technical details, please check our documentation at docs.techcorp.com or contact support@techcorp.com."Expected result: Responses with too many unverifiable claims are replaced with safe fallback messages
Log hallucination incidents for continuous improvement
Log hallucination incidents for continuous improvement
Add a logging step that records every flagged response — the user question, retrieved context, LLM response, and unverified claims. Store these in a database or send them to a monitoring tool. Review flagged responses weekly to identify patterns: are certain topics prone to hallucination? Does the knowledge base have gaps? Use this data to improve your vector store content and system prompt.
1// Code node for logging:2const logEntry = {3 timestamp: new Date().toISOString(),4 userQuestion: $('Webhook').first().json.message,5 retrievedChunks: $('Vector Store').first()?.json?.documents?.length || 0,6 llmResponse: $json.response,7 isVerified: $json.isVerified,8 unverifiedClaims: $json.unverifiedClaims,9 action: $json.isVerified ? 'passed' : 'flagged'10};1112return [{ json: logEntry }];13// Connect to a PostgreSQL node to INSERT INTO hallucination_logExpected result: A database of flagged responses enables continuous improvement of the knowledge base and prompts
Complete working example
1// ====== Post-Generation Hallucination Verifier — Code Node ======2// Place after the AI Agent / LLM node, before Respond to Webhook34const llmResponse = $input.first().json.output || $input.first().json.text || '';56// Get retrieved context from the vector store7const retrievedDocs = (() => {8 try {9 const docs = $('Vector Store').all() || [];10 return docs.map(d => (d.json.pageContent || d.json.content || '').toLowerCase());11 } catch {12 return [];13 }14})();1516const contextText = retrievedDocs.join(' ');17const responseLower = llmResponse.toLowerCase();1819// Extract factual claims from the response20const claimPatterns = [21 /\$[\d,.]+/g, // prices: $10, $1,00022 /\d+\.?\d*\s*%/g, // percentages: 99.9%23 /\d+\s*(gb|tb|mb|kb)/gi, // storage: 100GB24 /\d+\s*(days?|hours?|minutes?|seconds?)/gi, // durations25 /\d+\s*(users?|seats?|members?)/gi, // user counts26 /(?:version|v)\s*\d+\.\d+/gi // versions: v2.127];2829let allClaims = [];30for (const pattern of claimPatterns) {31 const matches = responseLower.match(pattern) || [];32 allClaims = allClaims.concat(matches);33}3435// Deduplicate claims36allClaims = [...new Set(allClaims)];3738// Verify each claim against context39const verified = [];40const unverified = [];4142for (const claim of allClaims) {43 const cleanClaim = claim.trim();44 if (contextText.includes(cleanClaim)) {45 verified.push(cleanClaim);46 } else {47 unverified.push(cleanClaim);48 }49}5051const UNVERIFIED_THRESHOLD = 2;52const isReliable = unverified.length < UNVERIFIED_THRESHOLD;5354return [{55 json: {56 response: llmResponse,57 verification: {58 isReliable,59 totalClaims: allClaims.length,60 verifiedCount: verified.length,61 unverifiedCount: unverified.length,62 unverifiedClaims: unverified,63 contextChunksUsed: retrievedDocs.length64 }65 }66}];Common mistakes when stopping a Language Model from Hallucinating Data in an n8n Chatbot
Why it's a problem: Relying solely on the system prompt to prevent hallucination without providing actual data via RAG
How to avoid: Always use a vector store retriever. The system prompt reduces hallucination tendency, but RAG provides the factual grounding
Why it's a problem: Setting temperature too high (0.7+) for a factual chatbot, encouraging creative generation
How to avoid: Set temperature to 0 for factual responses. Use 0.3-0.5 only for creative tasks like copywriting
Why it's a problem: Not updating the vector store when product information changes, causing stale answers
How to avoid: Build an automated pipeline that re-indexes your knowledge base whenever documentation is updated
Why it's a problem: Using Top K values that are too high (20+), flooding the context with marginally relevant content
How to avoid: Use Top K of 3-5 with a similarity threshold of 0.7 to ensure only highly relevant chunks reach the LLM
Best practices
- Use RAG with vector stores as the primary grounding mechanism — retrieve relevant data before every LLM call
- Set temperature to 0 for factual chatbots to minimize creative fabrication
- Write explicit anti-hallucination rules in the system prompt: 'ONLY use facts from CONTEXT'
- Include a specific refusal phrase for the model to use when it does not have information
- Add post-generation verification to catch numeric claims not present in source data
- Log flagged responses to a database for weekly review and knowledge base improvement
- Keep vector store content up to date — stale data causes the model to fall back on training knowledge
- Use chunk sizes of 500-1000 tokens with 100-token overlap for optimal retrieval accuracy
Still stuck?
Copy one of these prompts to get a personalized, step-by-step explanation.
My n8n chatbot is making up product features and prices that don't exist. How do I use RAG with a vector store, anti-hallucination system prompts, and post-generation verification to ground responses in my actual data?
Connect a Vector Store node (Pinecone/Qdrant/Supabase) to the AI Agent's retriever input with Top K: 5 and similarity threshold 0.7. Set the system prompt to 'ONLY use facts from CONTEXT. If the answer is not in CONTEXT, say I don't have that information.' Set temperature to 0. Add a Code node after the LLM to verify numeric claims against retrieved context.
Frequently asked questions
Can I completely eliminate hallucination from my AI chatbot?
No. Current LLMs cannot guarantee zero hallucination. However, RAG + strict system prompts + temperature 0 + post-generation verification reduces hallucination to near-zero for factual questions within your knowledge base. The key is having comprehensive source data and explicit refusal instructions for uncovered topics.
Which vector store should I use with n8n for anti-hallucination RAG?
For simplicity, use Supabase Vector (pgvector) since n8n has native Supabase support. For scale, use Pinecone or Qdrant. All three integrate with n8n's Vector Store node and produce equivalent grounding quality.
How do I know if my chatbot is hallucinating?
Add the post-generation verification Code node from this tutorial. It catches fabricated numbers, dates, and statistics. For semantic hallucination (correct format but wrong facts), review a sample of conversations weekly against your actual documentation.
Does lowering temperature to 0 make responses boring or repetitive?
Temperature 0 makes responses deterministic, which is ideal for factual chatbots. For the same question, you get the same answer every time. If you need variety in phrasing, use 0.1-0.2, which adds slight variation without significant hallucination risk.
How often should I update my vector store to prevent stale answers?
Re-index whenever source documentation changes. For active products, build an automated pipeline (n8n workflow triggered by webhook from your CMS) that re-embeds updated documents. Stale vector data is one of the top causes of hallucination in RAG systems.
Can RapidDev help me build a hallucination-free chatbot in n8n?
Yes. RapidDev builds production RAG pipelines in n8n with vector store setup, knowledge base indexing, anti-hallucination prompting, and verification layers for teams that need reliable, accurate AI chatbots.
Talk to an Expert
Our team has built 600+ apps. Get personalized help with your project.
Book a free consultation