The Chatbot Made Up Your Return Policy
A Lethbridge retailer I know set up a customer-facing AI chatbot last year. She trained it on some FAQs, pointed it at her website, and launched it. Within a week, a customer came back furious because the chatbot had told them they could return any item within 60 days. Her actual return window is 30 days.
The AI had not read her policy. It had guessed, based on what similar stores tend to do, and it guessed wrong. The policy was sitting right there on her website. The AI just never looked at it.
This is one of the most common frustrations people hit when they try to use AI for anything specific to their own business. The model knows a lot about the world in general. It knows almost nothing about your store, your team, or your rules.
Why AI Knows Nothing About Your Business
AI models learn from a large body of text collected up to a certain date. That text covers news, Wikipedia, books, forums, and a wide slice of the public internet. It does not include your employee handbook, your product catalogue, your pricing spreadsheet, or your service agreement.
When you ask the model something it was not trained on, it has two options. It can say it does not know. Or it can generate a plausible-sounding answer based on patterns from its training. It tends to do the second one, which is why AI sounds confident when it is wrong.
The fix is not to retrain the model on your documents. That process is expensive, time-consuming, and has to be repeated every time something changes. There is a better approach.
What RAG Actually Does
RAG stands for retrieval-augmented generation. The name is a mouthful, but the idea is straightforward. Instead of relying on the AI's training alone, the system retrieves relevant text from your documents first, then hands that text to the AI as context before asking it to answer.
Think of the difference between asking a new employee to answer a customer question from memory versus handing them the policy binder first and saying "the answer is in here." The AI is the employee. RAG is the binder.
The model does not need to memorize your documents. It just needs to be given the right passage at the right moment, and it becomes much better at giving you a correct, specific answer.
How It Works, Step by Step
The process runs in the background every time a user asks a question. Here is what happens.
A RAG system does three things in sequence:
- The question comes in. A search runs across your document library to find the passages most likely to contain the answer.
- The top matching passages are pulled and attached to the beginning of the prompt sent to the AI.
- The AI reads those passages and generates an answer based on what it actually found, not what it vaguely remembers from training.
The search step is the technical part. It uses a method called vector search (a way of measuring meaning, not just matching keywords) to find conceptually similar content even when the exact words do not match. A question about "what can I return" will still find a passage titled "Refund Eligibility."
A Real Example: Warranty Questions at an Auto Parts Counter
I worked with an Edmonton auto parts retailer who was getting the same questions at the counter every day. What is covered under warranty? How long is the warranty on this part? What do I need to bring in?
They had a 40-page warranty and returns policy document. Nobody had memorized it, and looking things up mid-conversation slowed everything down. We built a RAG-based assistant that had access to that document and a few product spec sheets.
Counter staff could type in a quick question and get a specific answer in seconds, with the relevant policy clause shown below the response. The AI was not guessing. It was reading the actual document and summarizing the relevant section.
The key detail: when the document was updated, the system immediately reflected those changes. No retraining, no delay.
Where RAG Works Well
RAG is well-suited to any situation where you have a body of documents and need to answer specific questions from them quickly.
Common use cases where RAG adds real value:
- Customer-facing FAQs grounded in your actual policies and product details
- Internal knowledge bases where staff can search procedures and manuals in plain English
- Contract review assistants that surface relevant clauses from a document archive
- Product support tools that pull from spec sheets and service bulletins
- Onboarding tools where new hires can ask questions against your internal documentation
In each of these cases, the value is that the AI is answering from your content, not from a general model's best guess.
What RAG Still Cannot Do
RAG is a meaningful improvement over a plain chatbot, but it is not magic. There are a few honest limits worth knowing before you build anything around it.
Your documents have to be reasonably organized and current. A RAG system built on outdated policies, poorly formatted PDFs, or inconsistent naming will produce inconsistent results. Garbage in, garbage out still applies.
RAG also does not help with tasks that require judgment, reasoning across many documents at once, or understanding business context that is not written down anywhere. If the answer to a customer question depends on a relationship history that lives only in your head, RAG cannot access that.
And like any AI system, it needs a human in the loop for high-stakes situations. A RAG assistant can surface the right warranty clause. It should not be the final word on a disputed $5,000 claim.
The Short Version
AI does not know your business by default. It knows the world. RAG is how you give it access to your specific documents so it can look things up instead of guessing.
The key things to remember:
- RAG retrieves relevant text from your documents and hands it to the AI as context before generating an answer
- It is better than retraining because updates are immediate and no technical rebuild is needed
- It works well for policies, FAQs, manuals, product specs, and internal knowledge bases
- It still depends on document quality, and retrieval errors can still produce wrong answers
- High-stakes decisions still need a human to review the output
If you have documents your team or customers ask questions about regularly, RAG is probably the right tool to look at. If you're curious whether it fits your situation, we do free discovery calls where we look at what you have and tell you honestly whether it makes sense.