NeurIPS 2023 Outstanding Paper

Are Emergent Abilities of Large Language Models a Mirage?

By Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo • Stanford University • 2023

"TL;DR: This paper argues that the sudden, unpredictable 'emergent abilities' observed in scaling large language models are not fundamental shifts in model capability, but rather artifacts of how researchers measure performance. When evaluated with linear or continuous metrics, performance scales smoothly and predictably."

Scaling Laws Metric Distortion AI Safety & Alignment BIG-Bench Meta-Analysis

Why This Matters

In recent years, the AI community became captivated by the concept of emergence. Prominent papers claimed that as Large Language Models (LLMs) grow in parameter count and training compute, they suddenly acquire capabilities—like multi-step arithmetic, translation, or logical reasoning—that were entirely absent in smaller models.

This narrative fueled both excitement and anxiety. In AI safety, it gave rise to the "sharp left turn" hypothesis: the fear that future, larger models might suddenly and unpredictably acquire dangerous, uncontrollable capabilities overnight without warning.

But what if these sudden leaps are just a statistical illusion?

The Big Idea

The core thesis is simple: emergent abilities are a mirage created by the choice of evaluation metrics.

When researchers evaluate models using nonlinear or discontinuous metrics (like Exact String Match or Multiple Choice Grade), a smooth, continuous improvement in the model's underlying token-level accuracy is compressed and warped. It appears as a flat line of "zero performance" followed by a sudden, explosive jump. If we switch to linear, continuous metrics (like Token Edit Distance or Brier Score), the illusion evaporates, revealing predictable, power-law scaling.

How It Works

Let's look at the mathematics of this distortion. Suppose a model's per-token cross-entropy loss $\mathcal{L}_{CE}$ scales smoothly as a power law with the number of parameters $N$:

$$\mathcal{L}_{CE}(N) = \left(\frac{N}{c}\right)^\alpha$$

Where $c > 0$ and $\alpha < 0$ are constants. The probability of the model generating a single correct token is then:

$$p(\text{single token correct}) = \exp(-\mathcal{L}_{CE}(N)) = \exp\left(-\left(\frac{N}{c}\right)^\alpha\right)$$

1 Mathematical Scaling Simulator

Interactive Demo

Adjust the sequence length ($L$) and scaling exponent ($\alpha$) below to see how a smooth token-level improvement transforms into a sharp, "emergent" step-function when evaluated using strict sequence-level accuracy.

Sequence Length ($L$): 5 tokens Length of the correct answer required (e.g., 5 digits in addition).

Scaling Exponent ($\alpha$): -0.2 Controls how fast the underlying model learns.

Accuracy ($p^L$) - Nonlinear

Token Edit Distance - Linear

2 The MNIST K-Subset Illusion

Interactive Vision Demo

The authors proved they could induce "emergent abilities" in standard vision models (which normally scale smoothly) by defining a metric called Subset Accuracy: requiring a model to classify $K$ independent images perfectly in a row to get a score of 1.

Require $K$ Correct Images: 3

Evaluation Sequence

Score = 1 if ALL are correct, else 0.

Single Image Accuracy (Smooth) Subset Accuracy (K-images)

3 Metric Swapper: Ablating Emergence

BIG-Bench Realizations

In the BIG-Bench meta-analysis, switching evaluation metrics on the exact same model outputs causes the "emergent ability" to disappear. Toggle below to witness how the curve changes.

Under Multiple Choice Grade, performance remains near-zero (random guess) until a threshold scale, simulating sudden emergence.

Results & Meta-Analysis

To test their hypothesis, the authors analyzed real-world benchmarks using the InstructGPT/GPT-3 family and performed a large-scale meta-analysis of BIG-Bench.

> 92%

of Claimed Emergent Abilities

on BIG-Bench occur under either Multiple Choice Grade or Exact String Match.

Emergence on Brier Score

When the continuous Brier Score is used, the exact same model outputs scale smoothly.

Above Chance

Smaller Model Capability

With larger test datasets (higher resolution), smaller models show clear, non-zero performance.

Limitations & Open Questions

While the paper presents a compelling and highly-awarded critique, it is important to note its boundaries:

Does not disprove emergence absolutely: The authors explicitly state they do not claim LLMs cannot have emergent abilities, but rather that existing claims are highly likely to be metric artifacts.
Complex Reasoning Chains: Some tasks requiring multi-step reasoning (like Chain of Thought) might still show non-linearities, though many argue these too can be broken down into smooth token-by-token generation steps.
Subjectivity of "Emergence": If a capability is only useful to a human when it is 100% correct (e.g., writing compiling code), then "functional emergence" from a human perspective might still feel sudden, even if the math underneath is smooth.

Glossary

Brier Score

A continuous, strictly proper scoring rule for assessing the quality of probabilistic predictions. It measures the mean squared difference between predicted probabilities and actual outcomes.

Token Edit Distance

A linear metric measuring the number of token insertions, deletions, or substitutions required to transform the model's output into the target string.

Scaling Laws

Empirical power-law relationships describing how a model's performance (usually cross-entropy loss) improves predictably as a function of parameters, dataset size, and compute.

Citation

@inproceedings{schaeffer2023emergent,
  title={Are Emergent Abilities of Large Language Models a Mirage?},
  author={Schaeffer, Rylan and Miranda, Brando and Koyejo, Sanmi},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2023}
}