We sent a prospect a price that does not exist.
An AI produced it. It sat in the middle of an email and looked like every other line around it: a plan, a number, a dollar sign. Nobody made that figure up on purpose, and nobody checked it, because nothing about it looked like it needed checking. It went out under our name, and someone weighing up whether to trust us with their business read a price we do not charge.
That is an AI hallucination, and it is the kind of mistake a small business cannot see coming. There is no version of this where we caught it in time.
The short version, if you read nothing else. These tools do not look something up and occasionally get it wrong. They produce the most plausible next words, and plausible is right often enough that you stop checking. Three gaps catch people out, and it is the same three every time: it cannot see what you have not shown it, it does not know today's prices, and it will fill a gap rather than leave one. About 10 seconds of checking, in the right place, catches nearly all of it.
What an AI hallucination actually is, and why "lying" is the wrong word
A hallucination is when it states something with confidence and the thing is not true. Not lying, not a bug, and worth understanding properly rather than dismissing. It is a property of how these tools work.
Lying needs an intention. A liar knows the true thing and picks the other one. Nothing like that happened here. The price arrived in the same tone, with the same confidence, in the same tidy formatting as a price that was real, because nothing inside the machine treats those two cases differently.
In practical terms that is worse than lying. A liar you learn to watch. This you cannot spot by reading it.
Why it happens, without the lecture
You don't need to know what is going on inside. One sentence does the job: it is built to produce the most plausible next words, not to know things.
Plausible and true overlap most of the time. That overlap is the entire reason these tools are useful, and it is also the trap. If it were wrong half the time you'd check everything and never get burned. Wrong once in a while is exactly what teaches you to stop looking.
Three gaps do nearly all the damage.
It cannot see what you have not shown it. Everything it can currently see is what the series calls its context, and that is the whole of its world. Anthropic's documentation for Claude Code spells out what falls outside it: live system state, anything beyond the material in front of it, business context it has not been given. Your price list, your diary, your policies, and the state a dog arrived in last June are not in there unless you put them in there.
It does not know today's anything. It learned from an enormous amount of text up to a point in time, and then it stopped. Prices, plan names, and product names move constantly, which is precisely why our price round-up carries the date we checked it. Ask what something costs and you are not querying a price list. You are asking something that has read a great many old price lists.
It will fill a gap rather than leave one. This is the one that got us. Asked for a price, it produced the shape of a price.
The vendors document this behaviour themselves. Anthropic's docs for Claude Code warn that an answer can sound assured and still be wrong. Google's page on its AI search modes says that where the response is not backed by enough confidence, you get shown a set of web links instead. If the companies building these things hedge in public, a groomer in the Inner West is allowed to hedge too.
The Last Look: the 10-second check that catches it
Every groomer already has this habit. Before the dog goes back to its owner you do a last look: nails, ears, the bit behind the elbow. It takes seconds, and it exists because the cost of missing something is out of all proportion to the cost of glancing.
Give an AI answer the same treatment before it leaves your hands. Three questions, in this order.
1. Did I show it? Run your eye over the answer and mark anything factual: a price, a date, a name, a policy line, a claim about a coat. Each one either came from something you pasted in, or it came from nowhere. There is no third option.
2. Could it possibly know? Anything that has changed lately, or that lives only in your own records, it cannot know. Your price list is in your head and your booking system. It is not in its training.
3. Who pays if this is wrong? Most of the time the honest answer is nobody, and you send it. Sometimes the answer is a client, a dog, or you. That is the moment you go and look it up.
That is the whole check, and it really does run in about 10 seconds once it's a reflex.
The reason it is three questions and not 12 is that a 12-point checklist gets done twice and then quietly abandoned. A last look gets done for the rest of your career.
The line you can add to any prompt
You can also make it show you its own seams. Paste this at the end of any prompt where the answer matters:
That does not make it accurate. Nothing you type makes it accurate. What it does is split the answer into the part you gave it and the part it supplied, so the second list is the only thing you have to go and verify. It can get that split wrong too, which is why it sits alongside the Last Look rather than instead of it.
When money is involved, be blunter still:
A visible gap is worth more than a number you cannot check. That placeholder is now how we draft anything with a price in it.
Where an AI hallucination costs a small business real money
Take Steph, a two-groomer salon in Sydney's Inner West. She is a worked example we follow through this series, and her numbers are the sort you would recognise rather than a real client's books.
A price you quote a client. Steph's casual full groom for a medium doodle is $95, the same figure we used working through maintenance grooming memberships. If a drafted reply says $85, because $85 is a perfectly plausible price for a groom, she has two options once it has gone: honour a price she never set, or open the relationship by correcting herself. Both cost more than the $10.
A policy you put in writing. The dangerous version is not a policy that is obviously wrong. It is a policy that sounds like yours, arrives in a tidy paragraph, and contains a 48-hour notice period you have never had. Whatever you send becomes the thing you have to argue from later, which is why writing your policies with AI and then getting them checked properly is its own job.
A claim about a breed or a coat. Moose is a Groodle who arrives matted every time, and Steph's de-matting surcharge is $30. An answer about how his coat behaves will land in exactly the same assured tone as everything else, and you won't be able to tell the difference by reading it. You know coats. It has read about coats. Those are not the same qualification.
Anything clinical. Not covered here, deliberately. A lump, a limp, a sudden change in behaviour, a medication question: that belongs to where AI must never go in pet care, written by Annika, who runs Hound Health Bondi. The short version is that the answer is a vet.
One more, and it is a different problem wearing similar clothes. The moment you paste a client's message in to get help with a reply, the question stops being "is this accurate" and becomes "what did I just hand over". That has its own rules, and they live in what you must never paste into an AI chatbot.
The rule we wrote afterwards
Our response was not "be more careful". Careful is not a system, and it fails on the day you're flat out.
We wrote a rule instead, and it is deliberately absolute: never assume, hardcode, or guess one of our own prices. Query the live source, every time, no exceptions. It lives in the instructions our own AI tools read before they start work, so the rule arrives before the mistake has a chance to.
You can do the same thing without any of our tooling. The reason it keeps getting your prices wrong is that it doesn't have your prices, and the fix for that is to hand them over and then refuse any number that came from somewhere else. Doing that once, properly, so you stop retyping yourself, is teaching it your prices, policies, and voice.
What this actually reclaims for you
Say you use it six times in a week: a couple of client replies, a caption, a policy tidy-up, a quote you were putting off. Six last looks is roughly a minute of your week.
Set that minute against one groom quoted at $85 when your price is $95, honoured because it went out in writing, and a client who from then on double-checks everything you tell her. That is not a close call.
What the minute reclaims is not really the minute. It is the version of the week where the tidy, assured, completely wrong answer is the one you sent. We paid for that lesson with a real price in a real email. You can have it for a minute a week.
Next in this series: Gemini, the free one already sitting in your Gmail. If you have landed here in the middle, the complete guide lays out the four levels and where this one sits.