Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
By Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, Denny Zhou · Google Research, Brain Team (2022)
"TL;DR: By prompting large language models with a few examples containing step-by-step intermediate reasoning paths (a 'chain of thought'), we can unlock state-of-the-art performance on complex arithmetic, symbolic, and commonsense reasoning tasks without any parameter updates."
Why This Matters
While scaling up language models (like GPT-3 or PaLM) has unlocked incredible capabilities in text generation, translation, and basic question answering, it has historically failed to improve performance on tasks requiring multi-step logical deduction. Math word problems, symbolic manipulations, and commonsense reasoning remained stubborn bottlenecks.
Prior approaches to solving this involved task-specific finetuning or generating formal mathematical representations. However, these methods are expensive, require massive human-labeled datasets of intermediate steps, and often result in brittle models that fail to generalize outside their narrow training distributions.
The Big Idea
Instead of just asking for the final answer, what if we show the model how to think?
The core contribution of this paper is Chain-of-Thought (CoT) Prompting. By augmenting standard few-shot exemplars with structured, intermediate natural language reasoning steps, we can leverage the model's pre-trained knowledge to decompose complex problems on the fly.
"A chain of thought is a sequence of natural language intermediate steps that lead to a final output. This mimics how humans solve multi-step problems: we write down scratchpad notes before arriving at a final conclusion."
How It Works
CoT Prompting works entirely in-context. It does not update any weights. Below is an interactive simulator where you can toggle between Standard Prompting and Chain-of-Thought Prompting to see exactly how the model input and output structures change across different tasks.
Key Results & Emergence
The paper's experiments across multiple benchmark suites revealed a striking phenomenon: Chain-of-Thought prompting is an emergent ability of model scale. It does not help (and can actually hurt) performance for smaller models (<10B parameters), but yields massive gains as models cross the 100B parameter threshold.
Scaling Curves: Standard vs CoT
Click datasets to watch the emergent performance gap open up.
Error Analysis & Scale Fixes
Why does scaling up fix reasoning? The authors manually categorized errors made by PaLM 62B and tracked how many of those exact failures were resolved when scaling to PaLM 540B.
Tracy used a piece of wire 4 feet long to support tomato plants in the garden. The wire was cut into pieces 6 inches long. How many pieces did she obtain?
She cut the wire into 6 inch pieces. This means she got 4 * 6 = 24 pieces.
The wire was 4 feet long. This means it was 4 * 12 = 48 inches long. It was cut into pieces 6 inches long. This means she obtained 48 / 6 = 8 pieces.
Limitations & Open Questions
- Is it actually reasoning? While the model mimics human step-by-step thinking, whether the neural network is truly "reasoning" or simply executing complex pattern matching over its pre-trained corpus remains an open philosophical and technical question.
- No guarantee of factual accuracy: A chain of thought can be highly coherent yet logically flawed or factually incorrect, leading to confident hallucinations.
- Cost of inference: Generating long reasoning sequences substantially increases the number of generated tokens, resulting in higher latency and compute costs during serving.
- Scale wall: Because CoT is an emergent property of models with ≥100B parameters, it is expensive and difficult to deploy in resource-constrained environments.
Glossary
Emergent Ability
A capability that is not present in smaller-scale models but appears suddenly as parameter count, dataset size, or training compute crosses a specific scale threshold.
Few-shot Prompting / In-Context Learning
The practice of providing a few input-output examples inside the prompt to demonstrate a task to the language model without modifying its underlying weights.
Out-of-Domain (OOD) Generalization
The model's ability to perform successfully on test examples that differ significantly in structure, length, or parameters from the examples provided in the prompt.
Cite this Paper
@inproceedings{wei2022chain,
title={Chain-of-thought prompting elicits reasoning in large language models},
author={Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and Ichter, Brian and Xia, Fei and Chi, Ed and Le, Quoc and Zhou, Denny},
booktitle={Advances in Neural Information Processing Systems},
volume={35},
pages={24824--24837},
year={2022}
}