Explore

Spurious Rewards: Rethinking Training Signals in RLVR

Rulin Shao, Shuyue Stella Li +12

We show that reinforcement learning with verifiable rewards (RLVR) can elicit strong mathematical reasoning in certain language models even with spurious rewards that have…

Attention Is All You Need — Explaining the Transformer

Ashish Vaswani, Noam Shazeer +6

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also…

RWKV: Reinventing RNNs for the Transformer Era

Bo Peng, Eric Alcaide +32

Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence…

Space Object Detection via Multi-frame Temporal Trajectory Completion

The Empowerment of Science of Science by LLMs

1-Shot RLVR: Reinforcement Learning for Reasoning with One Example

Yiping Wang, Qing Yang +12

We show that reinforcement learning with verifiable reward using one training example (1-shot RLVR) is effective in incentivizing the math reasoning capabilities of large…

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL

DeepSeek-AI, Daya Guo +198

General reasoning represents a long-standing and formidable challenge in artificial intelligence. Recent breakthroughs, exemplified by large language models (LLMs) and…

Denoising Diffusion Probabilistic Models (DDPM)

Jonathan Ho, Ajay Jain, Pieter Abbeel

We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium…

Language Models are Few-Shot Learners

Tom B. Brown, Benjamin Mann +29

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While…

Mem0: Scalable Long-Term Memory for AI Agents

Prateek Chhikara, Dev Khant +3

Large Language Models (LLMs) have demonstrated remarkable prowess in generating contextually coherent responses, yet their fixed context windows pose fundamental challenges for…

SkillOpt — Executive Strategy for Self-Evolving Agent Skills

Yifan Yang, Ziyang Gong +13

Agent skills today are hand-crafted, generated one-shot, or evolved through loosely controlled self-revision, none of which behaves like a deep-learning optimizer for the skill,…

RGSD: Rubric-Guided Self-Distillation

MohammadHossein Rezaei, Anas Mahmoud +7

Rubrics have emerged as an alternative to RLVR in open-ended domains where a single ground-truth final answer is not available. Existing rubric-based training methods rely on an…

Model Domain Mapping Function & Composition

Tülu 3: Pushing Frontiers in Open Language Model Post-Training

Nathan Lambert, Jacob Morrison +21

Language model post-training is applied to refine behaviors and unlock new skills across a wide range of recent language models, but open recipes for applying these techniques…

Kimi k1.5: Scaling Reinforcement Learning with LLMs

Kimi Team, Angang Du +94

Language model pretraining with next token prediction has proved effective for scaling compute but is limited to the amount of available training data. Scaling reinforcement…

The Entropy Mechanism of RL for Reasoning LLMs

Ganqu Cui, Yuchen Zhang +15

This paper aims to overcome a major obstacle in scaling RL for reasoning with LLMs, namely the collapse of policy entropy. Such phenomenon is consistently observed across vast RL…

Absolute Zero: Reinforced Self-play Reasoning with Zero Data

Andrew Zhao, Yiran Wu +9

Reinforcement learning with verifiable rewards (RLVR) has shown promise in enhancing the reasoning capabilities of large language models by learning directly from outcome-based…

Does RL Really Expand LLM Reasoning?

Yang Yue, Zhiqi Chen +6

Reinforcement Learning with Verifiable Rewards (RLVR) has recently demonstrated notable success in enhancing the reasoning performance of large language models (LLMs),…

Understanding R1-Zero-Like Training: A Critical Perspective

Zichen Liu, Changyu Chen +6

DeepSeek-R1-Zero has shown that reinforcement learning (RL) at scale can directly enhance the reasoning capabilities of LLMs without supervised fine-tuning. In this work, we…

DAPO: Demystifying Large-Scale LLM Reinforcement Learning

Qiying Yu, Zheng Zhang +33

Inference scaling empowers LLMs with unprecedented reasoning ability, with reinforcement learning as the core technique to elicit complex reasoning. However, key technical…

s1: Simple Test-Time Scaling

Niklas Muennighoff, Zitong Yang +8

Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance. Recently, OpenAI's o1 model showed this capability…

DeepSeekMath: Pushing the Limits of Mathematical Reasoning

Zhihong Shao, Peiyi Wang +9

Mathematical reasoning poses a significant challenge for language models due to its complex and structured nature. In this paper, we introduce DeepSeekMath 7B, which continues…

Dota 2 with Large Scale Deep Reinforcement Learning — OpenAI Five

OpenAI, : +25

On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as…

Decision Transformer: Reinforcement Learning via Sequence Modeling

Lili Chen, Kevin Lu +7

We introduce a framework that abstracts Reinforcement Learning (RL) as a sequence modeling problem. This allows us to draw upon the simplicity and scalability of the Transformer…