You Asked. It Answered. It Was Wrong.
A general contractor in Red Deer was putting together a permit application. He asked ChatGPT for the specific municipal requirement, and the AI gave him a full paragraph, complete with a permit category and a citation number. It looked right. It sounded authoritative.
He caught it before he submitted. The permit category it cited does not exist.
That kind of mistake has a name. People call it a hallucination. It is not a glitch, a bug, or a fluke. It is a predictable consequence of how AI actually works.
Understanding it will change how you use these tools.
AI Predicts Text. It Does Not Retrieve Facts.
When you type a question into ChatGPT or Claude, the AI does not search a database. It does not look up a stored answer. It generates a response, word by word, by predicting what text should come next based on patterns it absorbed during training.
Think of it as an extremely sophisticated autocomplete. You type "The capital of France is" and the word "Paris" follows because it is the most statistically likely next word. The AI is not consulting a map. It is doing pattern-matching on billions of examples of text it has seen before.
For most general questions, this works fine. The training data was full of accurate information on common topics, so the predictions line up with reality. The problem appears when you ask about something specific, recent, or rare.
The Confidence Comes From the Same Place as the Errors
Here is the part that catches most people off guard. The AI generates its answer the same way regardless of whether it actually "knows" the answer. A confident paragraph about photosynthesis and a confident paragraph about a law that was never passed look identical on the surface.
Same polished tone. Same complete sentences. Same complete absence of uncertainty.
The model was trained to produce coherent, fluent text. Fluency and accuracy are two different things, and training optimized heavily for the first one. There is no internal flag that says "I am guessing here." The output looks equally authoritative whether the underlying information is solid or invented.
That is not a design flaw that engineers missed. It is a consequence of how language models are built. Some newer systems have gotten better at hedging, adding phrases like "I am not certain about this" or "you may want to verify." But those qualifiers do not appear reliably, and they certainly do not appear every time they should.
The Kinds of Questions That Get You in Trouble
Some categories of question are far more likely to produce a wrong-but-confident answer. Knowing which ones they are is most of the battle.
Watch out when you are asking about:
- Local or regional specifics: municipal bylaws, permit categories, office addresses, local professional credentials, regional regulations
- Recent information: regulatory updates, current prices, role changes at organizations, news from the past year or two
- Citations and references: AI will invent plausible-sounding sources with confidence, including book titles that do not exist, cases that were never decided, and statistics with no origin
- Niche details in any field: if a topic was covered lightly in training data, the AI fills the gaps using patterns from nearby topics that may not apply
What connects all of these is specificity. The more specific your question, the more likely the AI is working from thin data and filling in gaps with pattern-matching. General questions about well-documented topics are safer. Specific questions about narrow, local, or recent subjects deserve more scrutiny.
It Does Not Know What It Does Not Know
The training data covered some topics thousands of times. Others showed up once or twice. When the AI encounters a gap, it does what it always does: it generates the most plausible-sounding continuation.
That continuation might be accurate, partially invented, or entirely made up. The model cannot reliably distinguish between those three outcomes.
Some AI tools have added explicit uncertainty signals, and those are worth paying attention to when they appear. But you cannot wait for the tool to flag itself before checking. A lot of wrong answers come through with no flags at all.
One Question Worth Asking Before You Use the Answer
Before you act on any AI answer, ask yourself: if this answer is wrong, what happens? If the stakes are low, a wrong answer costs you nothing. A first draft with a weak metaphor is easy to fix. A brainstorm list with one bad idea gets edited out.
If the stakes are higher, verify the answer independently. Higher-stakes situations include anything going into a contract, a permit application, a legal document, a compliance form, or anything presented to a client as factual. Any time a wrong detail would cost you money, time, or your reputation, check the specific facts against a primary source first.
You can also prompt the AI to surface its uncertainty. Try asking: "Are there any parts of this answer you are less confident about?" or "Flag anything here that I should double-check." It will not catch everything, but it brings shakier claims to the surface more often than they would otherwise appear.
This Is Not a Reason to Stop Using AI
Knowing about hallucinations is not an argument against using AI. It is an argument for using it in the right places. AI is genuinely excellent at tasks where rough accuracy is enough: drafting emails, summarizing long documents, generating ideas, formatting data, writing a first version of something you will edit.
It is less suited for tasks where a single wrong detail causes a real problem. That is not a failure of the technology. It is just a property of the technology, the same way a calculator is excellent at arithmetic but useless at reading a room.
The business owners who get the most out of AI are not the ones who trust it blindly. They are the ones who know which tasks it handles well and which tasks need a second look. That distinction takes about five minutes to learn and saves a lot of headaches.
The Short Version
AI generates text based on patterns. It does not retrieve facts. It sounds confident either way. Wrong answers and right answers look the same on the surface.
The more specific, recent, or local your question, the more carefully you should check the answer.
Use AI for the draft. Use it for the brainstorm. Use it for the repetitive work that does not require precision. For the specific detail that matters, check it yourself.
That is not distrust. That is just how you use the tool well.
If you want to talk through which tasks in your business AI can handle reliably and which ones need more care, we do free discovery calls. No pitch, just a straight conversation about your actual workflow.