AI hallucinations

An AI hallucination is when an AI system—especially a chatbot—states something false, made-up, or nonsensical with total confidence. It's the defining reliability problem of the current AI era, and (fittingly for this site) a brand-new industrial-scale source of things-that-sound-true-but-aren't.

What it looks like

• Invented citations: confidently citing books, papers, court cases or articles that don't exist — a notorious failure mode (lawyers have been sanctioned for filing AI-invented case law);
• Fabricated facts: made-up dates, statistics, quotes, biographies and events, delivered as fact;
• Fake details about real things: a real person with an invented award; a real book with an invented plot;
• Confident wrongness: the hallmark — the fluent, authoritative tone is identical whether the model is right or completely inventing.

Why it happens

• They predict, they don't 'know': large language models generate the most plausible next words based on patterns in training data — they're not looking up facts in a database. Plausible and true usually overlap, but not always;
• No built-in truth check: a base model has no inherent mechanism to verify a claim against reality — fluency is not the same as accuracy;
• Gaps get filled: asked about something rare or nonexistent, the model generates something that fits the pattern rather than admitting it doesn't know — strikingly like human confabulation;
• The confidence is a style, not a signal: models are trained to sound helpful and assured, so wrongness arrives dressed as expertise.

Protecting yourself

• Verify anything that matters — especially names, numbers, dates, quotes, citations, legal/medical/financial claims — against a primary source;
• Be extra suspicious of specifics: exact statistics, precise quotes and cited sources are the most common fabrications. If a citation exists, check that it exists;
• Ask for sources and then check them — don't trust that a cited link says what the AI claims (or exists at all);
• Treat AI as a fast, fallible assistant, not an oracle — brilliant for drafting and brainstorming, unreliable as a sole fact source;
• Notice your own bias: fluent, confident answers feel true — the same trap behind human misconceptions (spotting fakes).

The bigger picture

Hallucination is an active area of AI research, and newer systems (with retrieval, tool use, and better training) reduce it — but no current system eliminates it. The healthy stance is the same one this whole site preaches: confident and true are different things, whether the source is a rumor, your own memory, or a machine.