Hallucination — llm-kit

hallucination

// Why AI confidently says things that are completely wrong

💡 The core idea: AI doesn't "know" facts the way humans do. It learned to produce text that sounds correct based on patterns in training data. When asked about something rare, niche, or outside its training, it doesn't say "I don't know" — it fills in plausible-sounding text that may be completely false. This is called hallucination.

Examples of AI hallucinations

Underlined text = hallucinated (incorrect) information

Question: "Tell me about a paper by Dr. Zhang Wei on quantum computing from 2021"

In 2021, Dr. Zhang Wei published "Quantum Coherence in Neural Networks: A Theoretical Framework" in Nature Physics, demonstrating a 40% improvement in error correction.

⚠️ This paper does not exist. The AI invented the title, journal, and finding. Asking about a specific niche topic with a non-famous name is a recipe for hallucination.

Question: "What court cases support my argument about X?" (asked in a legal brief)

Relevant precedents include Johnson v. Massachusetts (2019), 234 F.3d 891, which established that... and Smith v. Google LLC (2022), which held...

⚠️ Famous real example: a lawyer submitted a brief with AI-generated fake case citations. None existed. The judge sanctioned the lawyer. Always verify legal citations from authoritative sources.

Question: "What are the side effects of combining Drug X and Drug Y?"

Combining these medications may cause serotonin syndrome in 23% of patients according to a 2020 FDA study. The mechanism involves inhibition of the MAO-B pathway...

⚠️ Medical hallucinations are dangerous. The statistics, study, and mechanism may all be fabricated. Always verify medical information with a licensed doctor and official sources.

Question: "What is the capital of Australia?" (basic fact, unlikely to hallucinate)

The capital of Australia is Canberra. ✓

✓ Well-known facts that appeared millions of times in training data are almost always correct. Hallucination risk is low for common knowledge.

Why AI Hallucinates

An LLM is not a search engine or a database. It's a pattern-completion machine. When given a prompt, it produces whatever text fits the pattern most naturally — not whatever is true. For common, well-documented facts, the pattern usually aligns with truth. For rare, niche, or recent information, it fills in plausible-sounding text that may be completely fabricated.

HIGH RISK: niche facts, citations, recent eventsLOW RISK: common knowledge, math