Stanford's 2024 retrieval research found that domain-specific RAG with curated chunks outperforms full fine-tuning on accuracy and freshness for nearly all customer-support and Q&A workloads.
RAG, not fine-tuning
When founders hear “train the AI on my website” they often imagine fine-tuning a model on their content. Skip that. Fine-tuning is slow, expensive, and most importantly stale: the moment you publish a new doc, your fine-tuned model is wrong.
Retrieval-augmented generation (RAG) is the modern approach. The model stays general; your content lives in a vector store; for each question the system retrieves the most relevant chunks and answers grounded in them. Update a doc, re-embed, and the AI is current within minutes. Every step below assumes RAG.
Pick the right sources
Four sources, in priority order: (1) help center, (2) pricing and product pages, (3) top 10 product-explainer blog posts, (4) last 50 resolved support tickets. Skip thought-leadership posts, careers pages, press releases, anything top-of-funnel. The training set should be tight and product-focused. Most teams that fail at AI chat hand the bot a 500-page corporate site and get vague, drifting answers.
Chunk by H2 section, not by character count
The default in most tools is “split every 500 characters with 50-char overlap”. Throw it out. Split per H2 instead. Each chunk should hold one topic end-to-end: full refund policy, full integrations list, full pricing tier description. The retriever matches against chunks, so chunks must be coherent topic units. Mixed-topic chunks produce confused answers.
Embed with source links attached
Every chunk stores its source URL alongside the embedding. When the AI answers, the source link goes in the answer (“Based on our pricing page: ...”). Two benefits: users trust answers more, and you can audit when the AI gets something wrong by clicking through to the chunk that backed the answer.
Set a retrieval similarity threshold
If the top retrieved chunk scores below your threshold (typically 0.75 cosine similarity), the AI must not answer. It escalates: “Let me get a teammate on this”, captures email, and queues a human reply. This is the single biggest hallucination guardrail. Without it, the AI invents answers from weak partial matches. With it, the AI says “I don’t know” honestly.
Re-index on schedule + on publish
Two re-indexing triggers: a weekly cron that re-embeds everything (catches edits and link rot), and a webhook on doc publish that re-indexes the changed page within minutes. The expensive failure mode is shipping a doc update and the AI quoting the old version for two weeks. Webhooks fix it.
Test the 20 real questions
Take the top 20 subjects from your last 90 days of tickets. Type each into the chat. Score each answer: correct, partially correct, wrong, escalated. Anything not in “correct” or “correctly escalated” is a fix. Usually the fix is editing the source doc, not tweaking AI settings. The AI is rarely the problem; the source content is.
Run weekly review, edit source docs, re-index
Friday: read failed AI conversations from the week. For each, identify the missing or wrong source content. Edit the doc, trigger re-index, verify the same question now answers correctly. After 4 weeks of this loop, the AI’s accuracy on top-100 questions plateaus around 90%, and the remaining 10% are correctly escalated.
The AI is rarely the problem. The docs are.
In 90% of cases where AI chat answers are wrong, the bug is in the source content, not the model. The AI gave the right chunk; the chunk was outdated, ambiguous, or contradicted another doc. The fix is editorial, not technical.
This is good news. It means your team has full control over chat quality without writing a line of ML code. Every Friday review session is a chance to upgrade the docs that the bot (and your help center, and Google) all read from. The AI is just the loud auditor of your knowledge base.
Frequently asked questions
Do I need to fine-tune a model to train an AI chatbot on my site?▼
No. Modern AI chatbots use retrieval-augmented generation (RAG): your content is chunked, embedded into a vector store, and the AI retrieves the most relevant chunks for each question and answers grounded in them. Fine-tuning is rarely necessary and usually a worse fit because your content changes weekly. RAG updates instantly when you re-index.
How much content do I need?▼
More than you think for breadth, less than you think for accuracy. Start with: your help center, pricing page, top 10 blog posts, and your last 50 resolved support tickets. That is usually 20 to 50 pages, which is plenty for the AI to answer 70% of repeat questions accurately. Add more only when you see specific gaps.
What's the right way to chunk content?▼
Chunk by H2 section, not by character count. A 500-character split frequently cuts in the middle of a topic, leaving half a refund policy stuck to half an integrations list. Splitting per H2 keeps one topic per chunk, which is what the retriever matches against. Source link plus chunk size of one section is the right unit.
Should I include the blog?▼
Selectively. Top-of-funnel posts can pollute the retrieval set with marketing language when the user asks a product question. Include only product-explainer and how-to posts, not pure thought-leadership. If a blog chunk is what the AI retrieves for “how do I reset my password?”, your training set is too broad.
How do I keep training fresh?▼
Re-index on a schedule (weekly is fine for most teams) and on every doc publish via webhook. The expensive failure mode is shipping a doc update and the AI quoting the old version for two weeks. Re-index also when you fix a wrong AI answer: edit the doc, trigger re-index, verify in chat. Five-minute loop.
How do I prevent hallucinations?▼
Three guardrails: (1) tight retrieval threshold so the AI does not answer when no chunk matches well, (2) explicit handoff when below threshold (“let me get a teammate”), (3) source citations on every answer so users can verify and you can audit. Hallucinations almost always trace back to weak retrieval, not the language model itself.
Keep reading
Last updated: May 1, 2026