// How we measure if an AI is actually getting smarter (and why benchmarks lie)
💡 The core idea: How do you compare two AI models? Give them a standardized test. Like SATs for humans — a fixed set of questions that everyone answers the same way. A model that scores 90% on MMLU answered 90% of a wide range of knowledge questions correctly. But just like SAT scores don't capture everything, no single benchmark tells the whole story.
The most important AI benchmarks explained
Approximate scores as of early 2025. Scores shift as models update.
MMLU is the most common benchmark
MMLU (Massive Multitask Language Understanding) tests 57 subjects including mathematics, history, law, medicine, physics, ethics, and more — 14,000+ multiple-choice questions. A random guesser scores 25%. Humans average ~89%. GPT-4 scores ~87%. Modern frontier models now exceed 90%, approaching the ceiling of this benchmark.