Scaling Instruction-Finetuned Language Models
Authors: Hyung Won Chung*, Le Hou*, Shayne Longpre*, Barret Zoph†, Yi Tay†, William Fedus†, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, Jason Wei*
TL;DR
This paper explores instruction finetuning at scale, combining model size (up to 540B parameters), task count (up to 1,836 tasks), and multi-step reasoning data. The resulting models, Flan-PaLM and Flan-T5, achieve state-of-the-art performance across challenging reasoning benchmarks while requiring only a fraction of the original pre-training compute.
Why This Matters
Pre-trained language models are incredibly powerful, but they are trained primarily to predict the next word on massive web corpora. This objective makes them prone to repeating prompts, generating endless loops, or failing to directly answer simple instructions.
While instruction finetuning helps align models with human intent, previous attempts were limited in scale. They either scaled the model size or the number of tasks, but rarely both. Crucially, early instruction-finetuned models struggled with multi-step reasoning: finetuning on simple tasks actually degraded their ability to solve complex problems using Chain-of-Thought (CoT) prompting.
The Big Idea
The authors introduce Flan (Finetuning Language Models), demonstrating that instruction finetuning scales exceptionally well when you systematically scale three dimensions simultaneously:
- Scale the number of tasks: Up to 1,836 tasks drawn from 473 datasets across 146 task categories.
- Scale the model size: Testing models ranging from tiny 80M parameter T5 architectures up to a massive 540B parameter PaLM model.
- Incorporate reasoning data: Interweaving a small subset of Chain-of-Thought (CoT) datasets into the training mixture to protect and unlock reasoning capabilities.
The result is a unified finetuning recipe that dramatically improves zero-shot, few-shot, and CoT reasoning, making models more robust, usable, and safe.
How It Works
The Flan recipe combines diverse task mixtures and formats them creatively to support both zero-shot and few-shot paradigms. Let's explore the core ingredients.
1. The 1.8K Task Mixture
The training data is combined from four major collections: Muffin (tasks, dialog, and code), T0-SF (T0 tasks without evaluation overlaps), Natural Instructions v2 (NIV2), and a custom CoT Reasoning mixture.
Interactive Prompt Formatter
Configure how prompts are formatted for Flan models compared to base models.
2. Joint Scaling of Tasks & Parameters
Does instruction finetuning scale infinitely? The paper shows that both model scale and task count are crucial. While adding tasks past 282 shows diminishing returns, the gap between finetuned and baseline models remains massive across all scales.
Scaling Curve Explorer
Hover over the data points to see the normalized average score across held-out tasks.
3. CoT Finetuning: The Crucial Ingredient
Finetuning only on non-CoT tasks actually harms reasoning. By integrating just 9 manual CoT datasets into the mixture, the model retains its reasoning capability and unlocks zero-shot reasoning via the trigger phrase "Let's think step-by-step".
Zero-Shot CoT Showdown
Compare how the base model fails on complex tasks compared to Flan-PaLM.
Key Results & Benchmarks
The authors evaluated Flan across several challenging benchmarks, including MMLU, BIG-Bench Hard (BBH), TyDiQA, and MGSM.
| Model | MMLU (Direct) | BBH (CoT) | TyDiQA (Direct) | MGSM (CoT) |
|---|---|---|---|---|
| PaLM 540B (Pretrained) | 71.3% | 63.7% | 52.9% | 45.9% |
| Flan-PaLM 540B | 73.5% | 66.3% | 67.8% | 57.0% |
| T5-XXL 11B (Baseline) | 25.9% | 19.3% | 0.0% | 1.0% |
| Flan-T5-XXL 11B | 55.1% | 41.4% | 19.0% | 4.9% |
Limitations & Open Questions
Despite its impressive performance, the authors highlight several key limitations:
- Diminishing Returns on Tasks: Scaling from 282 tasks to 1,836 tasks showed marginal gains. Is this due to task redundancy, or is there a fundamental bottleneck in instruction following?
- Sociotechnical Biases: While toxicity is reduced, Flan models still propagate cultural biases and gender role assumptions present in the training corpora (as seen in translation evaluations).
- Inability to Self-Correct in Zero-Shot: Pre-trained models can still generate hallucinations or repeat instructions when faced with highly ambiguous tasks.
Glossary
Instruction Finetuning
The process of taking a pre-trained language model and training it further on a collection of diverse datasets formatted explicitly as instructions (e.g., "Summarize the following text: ...").
Chain-of-Thought (CoT) Prompting
A prompt engineering technique that encourages a large language model to output a step-by-step reasoning process before providing the final answer, dramatically improving performance on math and reasoning tasks.
Self-Consistency (SC)
An advanced decoding strategy where the model samples multiple different reasoning paths (CoT) and selects the final answer that appears most frequently among all generated paths.
Citation
@article{chung2022scaling,
title={Scaling Instruction-Finetuned Language Models},
author={Chung, Hyung Won and Hou, Le and Longpre, Shayne and Zoph, Barret and Tay, Yi and Fedus, William and Li, Yunxuan and Wang, Xuezhi and Dehghani, Mostafa and Brahma, Siddhartha and others},
journal={arXiv preprint arXiv:2210.11416},
year={2022}
}