AI hallucinations
Updated 2026-09-03What this is
A large 2025 study measured it directly: across 576,000 generated samples, roughly one in five suggested packages did not exist. The same failure appears inside real libraries as invented exports and invented methods, because the model completes what a plausible API would look like, not what the installed one is.
Why it matters
The cheap failure is a crash: the import is undefined, the method throws. The expensive failure is the package manager: an invented name that reads plausibly is exactly what gets typo-registered, and installing it hands the squatter a postinstall script on your machine. The study found invented names were consistent across re-prompts, which makes them targetable.
What the probe looks for
Closed vocabularies only, where absence is a fact rather than a guess: named imports from react outside its real export list, named imports from next/navigation outside its list, and Prisma model methods outside the client's fixed set. Suggested-but-nonexistent npm packages are deliberately out of scope for a client-side scanner; the supply-chain probe owns the squatting patterns that follow from them.
RELATED PROBES
- · AI Hallucinations