Instruction Tuning
cs.CL · 35 papers
Training Compute-Optimal Large Language Models
MMLU: 67.6%Jordan Hoffmann, Sebastian Borgeaud +20
We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. We find that current large language models are…
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
GSM8K: 57.0%Jason Wei, Xuezhi Wang +7
We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex…
Attention Is All You Need — Explaining the Transformer
BLEU: 28.4Ashish Vaswani, Noam Shazeer +6
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also…
Mem0: Scalable Long-Term Memory for AI Agents
LOCOMO: 67.13%Prateek Chhikara, Dev Khant +3
Large Language Models (LLMs) have demonstrated remarkable prowess in generating contextually coherent responses, yet their fixed context windows pose fundamental challenges for…
SkillOpt — Executive Strategy for Self-Evolving Agent Skills
best-or-tied evaluated cells: 52/52 cellsYifan Yang, Ziyang Gong +13
Agent skills today are hand-crafted, generated one-shot, or evolved through loosely controlled self-revision, none of which behaves like a deep-learning optimizer for the skill,…
RWKV: Reinventing RNNs for the Transformer Era
HellaSwag: 74.8%Bo Peng, Eric Alcaide +32
Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence…
Tülu 3: Pushing Frontiers in Open Language Model Post-Training
GSM8K: 87.6%Nathan Lambert, Jacob Morrison +21
Language model post-training is applied to refine behaviors and unlock new skills across a wide range of recent language models, but open recipes for applying these techniques…
Does RL Really Expand LLM Reasoning?
Omni-MATH-Train: 42.5%Yang Yue, Zhiqi Chen +6
Reinforcement Learning with Verifiable Rewards (RLVR) has recently demonstrated notable success in enhancing the reasoning performance of large language models (LLMs),…
Self-Instruct: Aligning LMs with Self-Generated Instructions
ROUGE-L: 39.9Yizhong Wang, Yeganeh Kordi +5
Large "instruction-tuned" language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless,…
GPQA: A Graduate-Level Google-Proof Q&A Benchmark
GPQA Main Set: 71.9%David Rein, Betty Li Hou +6
We present GPQA, a challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. We ensure that the questions are…
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
SWE-bench: 1.96%Carlos E. Jimenez, John Yang +5
Language models have outpaced our ability to evaluate them effectively, but for their future development it is essential to study the frontier of their capabilities. We find…
Toolformer: Language Models Can Teach Themselves to Use Tools
ASDiv: 40.4%Timo Schick, Jane Dwivedi-Yu +6
Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle…
Constitutional AI: Harmlessness from AI Feedback
HHH Eval: 180Yuntao Bai, Saurav Kadavath +49
As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through…
Aligning AI: Inside OpenAI's InstructGPT & RLHF Pipeline
API prompt distribution: 85 ± 3%Long Ouyang, Jeff Wu +18
Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful,…
Scaling Instruction-Finetuned Language Models — Flan-PaLM & Flan-T5
MMLU: 75.2%Hyung Won Chung, Le Hou +33
Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we…
Weak-to-Strong Generalization: Eliciting Strong Capabilities with Weak Supervision
NLP Benchmarks: nearly 80%Collin Burns, Pavel Izmailov +10
Widely used alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on the ability of humans to supervise model behavior - for example, to evaluate…
Qwen3 Technical Report: Unified Reasoning & Dynamic Thinking Budgets
AIME'24: 85.7An Yang, Anfeng Li +58
In this work, we present Qwen3, the latest version of the Qwen model family. Qwen3 comprises a series of large language models (LLMs) designed to advance performance, efficiency,…
DeepSeek-V3 Technical Report
MMLU-Pro: 75.9%DeepSeek-AI, Aixin Liu +198
We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. To achieve efficient inference and…
Gemini 1.5: Unlocking Multimodal Long-Context
Synthetic Text Haystack: >99.7%Gemini Team, Petko Georgiev +1135
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning…
Gemini: A Family of Highly Capable Multimodal Models — Deep Dive
MMLU: 90.04%Gemini Team, Rohan Anil +1349
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family…
PaLM: Scaling Language Modeling with Pathways
Model FLOPs Utilization (MFU): 46.2%Aakanksha Chowdhery, Sharan Narang +65
Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number…
Mixtral of Experts: Deep Dive
MMLU: 70.6%Albert Q. Jiang, Alexandre Sablayrolles +24
We introduce Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) language model. Mixtral has the same architecture as Mistral 7B, with the difference that each layer is composed of…
Mistral 7B: Engineering Extreme Efficiency in LLMs
MMLU: 60.1%Albert Q. Jiang, Alexandre Sablayrolles +16
We introduce Mistral 7B v0.1, a 7-billion-parameter language model engineered for superior performance and efficiency. Mistral 7B outperforms Llama 2 13B across all evaluated…
Llama 2 Explained: Pretraining, Fine-Tuning, and Safety Alignment
MMLU: 68.9%Hugo Touvron, Louis Martin +66
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our…