This transcript comes from automatic captions and may contain small errors.
0:00 IntroAsk an AI assistant a question and you get an answer.
Fluent, confident, and nicely formatted.
Most of the time it is right.
But sometimes it is completely made up and it sounds exactly the same.
This is called hallucination.
In this video, we look at why it happens, how a model can learn to know what it does not know, and how you can protect yourself.
In 2023, lawyers in New York submitted a brief with court cases that ChatGPT had invented.
The judge fined them $5,000.
In 2024, Air Canada's chatbot invented a refund policy.
A tribunal ruled that the airline was responsible for what its chatbot said.
In 2025, Deloitte partially refunded the Australian government for a report with fabricated references and a made-up quote from a court ruling.
0:45 Why models make things upAnd in Europe, the privacy group Noyb, short for none of your business, filed a complaint after ChatGPT falsely described a Norwegian man as a murderer of his own children.
Under the GDPR, personal data has to be accurate.
A researcher tracking court cases has now cataloged well over a thousand where AI made up legal sources.
To understand why, remember what a language model is trained to do.
Continue text in a way that sounds right.
Sounding right and being right usually go together, but not always.
Some facts, like a person's birthday or the title of an obscure paper, may appear only once in all the training data or never.
1:30 Guessing and benchmarksThere is no pattern to learn them from.
Ask about one of those and the model can still produce something with the right shape.
A plausible date, a plausible title.
The form is right.
The fact is invented.
A 2025 paper from OpenAI researchers adds a second reason.
Think of a multiple-choice exam.
If you leave a question blank, you get zero points.
If you guess, you might get lucky.
So, the smart strategy is to always guess.
Most AI benchmarks are graded the same way.
Saying I don't know scores zero.
So, models are trained to be good test takers and confident guessing wins.
2:15 CalibrationOn one fact quiz, an older model almost never declined to answer and was wrong three quarters of the time.
A newer one declined about half the time and was wrong only about a quarter of the time with almost the same number of correct answers.
The researchers' suggestion, change the scoring so that a confident wrong answer costs more than an honest I don't know.
The goal is not a model that is never uncert.
It is a model whose confidence matches reality.
This is called calibration.
Think of a good weather forecaster.
On the days they say 70% chance of rain, it rains about 70% of the time.
For a model, it means when it is 90% sure, it should be right about nine times out of 10.
3:00 Telling when it guessesWhen it is unsure, it should say so.
Encouragingly, research suggests large models often do carry this information inside.
In 2022, Anthropic researchers found that models mostly know what they know and can estimate whether their own answers are true.
But training can disturb it.
In the GPT-4 report, the pre-trained model was well calibrated and after training to be a helpful assistant, its calibration got worse.
So, can we tell when a model is guessing?
One clever method is simply to ask the same question several times.
If the model knows the answer, it says the same thing each time, perhaps in different words.
If it is guessing, the answers wander.
A different year, a different name each time.
3:45 Retrieval and citationsResearchers at Oxford turned this into a measure called semantic entropy, published in nature in 2024.
When the meanings disagree, the answer is likely made up.
And interpretability research has found a switch inside a model that decides whether it recognizes something.
When that switch fires by mistake, a hallucination can follow.
The most practical fix is to stop asking models to answer from memory.
With retrieval, the system first searches for relevant documents, then the model answers using them.
It turns a closed book exam into an open book exam.
And with citations, the model shows where each claim came from, so you can click and check.
4:30 Limits of retrievalBut retrieval is not a magic cure.
A 2024 Stanford study tested legal research tools built on retrieval and marketed as reliable.
They still hallucinated in roughly one in six to one in three queries.
In a 2025 test of AI search engines, researchers gave them a quote from a news article and asked for the source.
They got it wrong more than 60% of the time, often with full confidence.
A citation is a promise you can check.
It is not a proof that the claim is true.
So, what does an honest model look like?
It says I don't know when it doesn't.
It shows how sure it is.
It cites sources that actually support the claim, and it never invents a quote, a case, or a reference.
5:15 How to protect yourselfGetting there means training and testing for honesty, not just for accuracy, and rewarding a model for admitting uncertainty instead of punishing it.
Until then, a simple rule, never trust an AI answer just because it sounds confident.
Check the sources.
Be extra careful with names, numbers, dates, quotes, and references.
These are exactly where models invent things.
Click the link, read the passage, make sure it actually says what the answer claims.
Ask, "How sure are you?"
and "Where does this come from?"
And when it really matters for health, law, or money, confirm it with a person or an original source.
6:00 ClosingUse AI as a brilliant first draft, not as the final word.
The wisest people are not the ones who know everything.
They are the ones who know the limits of what they know.
We should ask the same of our machines and of ourselves.
Sometimes the most trustworthy answer is, "I don't know.
Let's check."