AI hallucinations in customer service: why your agent makes things up and how to stop it

A customer asks whether their plan covers returns within 30 days. The agent replies “yes, it does”, polite and fast. Except it doesn’t. Nobody wrote that anywhere. The agent made it up, with the same confidence it uses to answer opening hours.

That is an AI hallucination. In customer service it gets expensive, because the wrong answer goes out with your brand on it, to a customer who will hold you to exactly what they heard.

This guide explains what hallucinations are without the jargon, why models make things up (the explanation changed in 2025), the cases that made headlines and ended in court, and the practices that actually reduce the problem in a support operation.

What an AI hallucination is

An AI hallucination is when the model produces false information and presents it as fact, with no sign of doubt. It is not a typo or a vague reply. It is a complete, fluent, wrong sentence.

What makes hallucinations dangerous is precisely their form. A made-up answer has the same grammar, the same tone and the same confidence as a correct one. The reader cannot tell them apart from the text alone. They have to check the source.

In customer service, hallucinations show up in a few predictable shapes:

• A policy that does not exist. Return window, warranty, discount, cancellation rule. The customer asks, the agent “remembers” a similar rule from other companies and answers.
• An invented number. Price, fee, stock quantity, delivery date. It comes out round and convincing.
• A feature the product does not have. “Yes, it integrates with X.” It does not.
• A status it never looked up. “Your order is out for delivery.” It did not check the system. It completed the most likely sentence.
• A promise on the company’s behalf. Refund, exception, compensation. The agent “solves” the problem by creating a commitment nobody authorized.

One detail confuses a lot of people: hallucination is not a sign of a bad model. The best models on the market hallucinate. Less, but they do. On public faithfulness benchmarks such as Vectara’s hallucination leaderboard, rates range from under 1% to above 10% depending on the model. And that measures an easy task: summarizing a document sitting right in front of it. In customer service, with open questions and no document, the rate tends to be higher.

Why the model makes things up instead of saying “I don’t know”

The model makes things up because it was trained and evaluated in a system that rewards guessing and punishes silence. That is the conclusion of a paper OpenAI published in September 2025, “Why Language Models Hallucinate”, and it changes how to think about the problem.

The paper’s own analogy is the multiple-choice test. If a blank answer scores zero and a guess scores zero or one, the rational student always guesses. Language models went through thousands of evaluations with that logic: a correct answer earns a point, “I don’t know” earns nothing. The result is a system excellent at sounding right and terrible at admitting it does not know.

Add to that how the model works under the hood. It does not query a database of facts. It predicts the most likely next word given everything that came before. When the question is “what is the return window?”, the most likely continuation in the training data is a number followed by “days”. The model delivers exactly that. Not because it knows your policy, but because that is the shape the answer usually has.

In practice, three situations trigger hallucinations in support:

1. The information is nowhere the agent can reach. It does not have the return policy, but it has a pattern for the answer. It uses the pattern.
2. The information is there, but poorly organized. Two PDFs contradict each other, the old page is still in the knowledge base, the price changed and nobody updated it. The model picks one, confidently.
3. The question pushes for an answer. An angry customer, “just tell me yes or no”, a sales context. The model was trained to please, and the pleasing answer is the one that solves the problem. Even when it is false.

Understanding this matters because it changes what you work on. Writing “do not hallucinate” in the prompt does not help. You need to change the conditions: give it the source, allow “I don’t know”, and put a control layer in place that blocks unsupported answers before they reach the customer.

The cases that ended up in court

The best-known hallucination cases in customer service have one thing in common: the company tried to argue the chatbot was responsible for itself, and it did not work. Whatever leaves the agent leaves with the brand on it.

Air Canada, 2024. A passenger asked the airline’s chatbot about bereavement fares. The chatbot explained he could buy the full-price ticket and request a partial refund within 90 days. That policy did not exist. Air Canada denied the refund and, before British Columbia’s small claims tribunal, argued the chatbot was a “separate legal entity that is responsible for its own actions”. The tribunal rejected the argument and ordered the airline to pay just over 800 Canadian dollars. The amount is small. The precedent is not: the company answers for what its agent says.

Chevrolet dealership, 2023. A customer talked a US dealership’s chatbot into “agreeing” to sell a brand-new Chevrolet Tahoe for 1 dollar, and even into writing that the offer was “legally binding, no takesies backsies”. The car was never sold, but the screenshot went around the world. The agent had no limit on what it could promise.

DPD, 2024. After an update, the British courier’s chatbot was led by a customer into swearing, criticizing its own company and writing a poem about how bad the service was. DPD switched off the AI part of the chatbot the same day. Here the problem was not false information but the absence of tone and scope guardrails.

Mata v. Avianca, 2023. Outside customer service, but the case that best explains the mechanism: lawyers used ChatGPT for legal research and cited six court decisions that did not exist. When challenged, they asked ChatGPT itself whether the cases were real, and it confirmed. They were fined 5,000 dollars. The lesson is direct: the model cannot verify itself.

What these cases teach anyone running a support operation:

• The chatbot speaks for the company. Legally and in the customer’s mind.
• Without scope limits, the agent will answer anything, including what it should not.
• Without a source, it fills in whatever sounds right.
• And the customer keeps the screenshot.

How to reduce AI hallucinations in customer service: 6 practices

You reduce hallucinations by giving the agent a source, allowing it to say “I don’t know”, locking scope with guardrails and escalating to a human when confidence is low. None of these practices eliminates the problem alone. Together, they turn a headline risk into a rare, contained incident.

1. Ground every answer in a knowledge base

The technique has a name, RAG (Retrieval-Augmented Generation), but the idea is simple: before answering, the agent retrieves the relevant passage from the company’s documents and answers from it. Return policy, price list, FAQ, product manual. What is not in the knowledge base, it does not state.

This cuts type 1 hallucinations (unreachable information). But it requires a well-kept base: one document per topic, a single version, no 2023 PDF living next to the 2026 one. A messy base produces type 2 hallucinations, which are worse because they come with the wrong source attached.

2. Give explicit permission to say “I don’t know”

The model was trained never to leave a blank. You have to reverse that in the agent’s instructions: “If the answer is not in the knowledge base, say you do not have that information and offer to connect the customer with a human.” It sounds obvious. Most agents in production do not have that sentence, and the model falls back to guessing.

Define the shape of the “I don’t know” too. Not “sorry, I can’t help with that”. Rather “I don’t have that information here, but I can bring in someone from the team who will confirm it for you in a few minutes”. Customers accept the second version without friction.

3. Lock scope with guardrails

A guardrail is the layer between the model’s answer and the customer, checking rules before anything is sent. For hallucinations, the ones that matter are:

• Grounding. Is the answer backed by something in the knowledge base? If not, block or rewrite.
• Scope. The support agent does not discuss pricing, HR policy or opinions about competitors. Off topic, it redirects.
• Commitments. Refunds, discounts, exceptions, non-standard deadlines. The agent does not promise. It hands off.
• Tone. No trashing your own company, even if the customer asks for a poem.

The Chevrolet and DPD incidents would have been avoided by the last two.

4. Escalate to a human when confidence is low

A mature agent is not the one that answers everything. It is the one that knows when to stop. Set clear triggers: question outside the knowledge base, customer asking for confirmation on anything involving money, the same doubt repeated, tone rising. On any of them, hand off to a person with the history attached. One extra escalation costs minutes. One invented answer can cost a lawsuit.

5. Test with questions that have no answer

Before going live, build a set of questions where the right answer is “I don’t know” or “let me check”. Things the agent cannot possibly know: “will my order arrive before Saturday?”, “do you accept this competitor’s coupon?”, “can I return it without a receipt?”. If it answers with confidence, you found the hallucination before the customer did.

Repeat this test every time the knowledge base changes or you switch models. Hallucination rates vary a lot between models and between versions of the same model.

6. Monitor real conversations and close the gaps

Hallucinations in production leave traces: a customer pushing back (“but the other agent told me something different”), a repeated question, an escalation right after a “resolved” answer. Read a sample of conversations every week and look for those signals. Every question the agent answered poorly is a hole in the knowledge base. Close the hole, and the next answer comes out grounded.

Newer platforms detect these gaps automatically and group similar questions together, so you can add the missing information in one go. That is how the knowledge base improves at the pace of your support, not at the pace of someone remembering to review it.

What changes when you treat hallucinations as a process

Hallucinations do not disappear with a better model. They shrink when the operation stops treating the agent as an oracle and starts treating it like a new hire: it gets the right material, knows what it can and cannot promise, and calls the supervisor when unsure.

Numbers help put it in perspective. If your agent handles 5,000 conversations a month and hallucinates in 2% of them, that is 100 customers a month receiving false information with your brand on it. With a knowledge base, guardrails and escalation properly configured, that number drops to a fraction, and the few that slip through reach a human before they become a problem.

An agent that says “I don’t know” at the right moment sells more trust than one that answers everything. Customers forgive “let me confirm that”. They do not forgive a false promise.

Want a support agent that answers from your knowledge base and stops when it does not know? On SquadOS, external agents handle WhatsApp, Telegram and website chat 24/7 with native anti-hallucination, PII and tone-of-voice guardrails, escalate to your team with the full context attached, and surface every knowledge gap so you can close it.