ReAct: Synergizing Reasoning and Acting in Language Models

Human intelligence seamlessly combines task-oriented action with internal verbal reasoning. When cooking, we don't just execute physical commands; we reason internally ($“\text{now that the water is boiling, I should add the pasta}”$), track our progress, and adapt to exceptions ($“\text{I don't have salt, let me use soy sauce instead}”$).

In artificial intelligence, these two capabilities have historically been split:

• Reasoning-only models (like Chain-of-Thought) operate as closed-loop, static black boxes. They rely entirely on internal weights, leading to factual hallucinations and error propagation.
• Acting-only models (like web navigation or robotic controllers) generate direct action plans but lack abstract, high-level goal tracking, making them brittle when environments change unexpectedly.

ReAct bridges this gap, establishing a unified paradigm where language models generate both reasoning traces and actions in an interleaved sequence.

The core formulation of ReAct is beautifully simple. Consider an agent interacting with an environment. At step $t$, the agent receives an observation $o_t$ and takes an action $a_t$ based on its policy $\pi(a_t | c_t)$, where $c_t = (o_1, a_1, \dots, o_t)$ is the interaction history.

ReAct augments this action space to $\hat{\mathcal{A}} = \mathcal{A} \cup \mathcal{L}$, where $\mathcal{L}$ is the space of free-form natural language. An action $\hat{a}_t \in \mathcal{L}$ is a thought (or reasoning trace). Because thoughts do not affect the external environment, they yield no immediate observation feedback. Instead, they update the agent's internal context:

This allows the model to perform dynamic planning, track subgoals, inject commonsense knowledge, and gracefully handle errors.

Depending on the task, ReAct utilizes different execution patterns:

1. Knowledge-Intensive Tasks (e.g., HotpotQA, FEVER): The model alternates strictly between generating a thought, executing an action (like search or lookup), and receiving an observation. This prevents hallucination by anchoring every reasoning step to verified facts.
2. Interactive Decision Making (e.g., ALFWorld, WebShop): The model generates thoughts sparsely. It decides autonomously when to pause and reason about high-level planning, subgoals, or error recovery before continuing with environmental actions.

ReAct was evaluated across four diverse benchmarks. The results demonstrate that while standalone reasoning (CoT) and acting (Act) have severe limitations, their synergy yields state-of-the-art performance.

While powerful, ReAct is not without its failure modes:

• Rigid Structural Constraints: The strict interleaving of thought, action, and observation reduces the model's flexibility compared to pure free-form reasoning, occasionally leading to higher reasoning error rates.
• Search & Retrieval Vulnerability: If the external knowledge source returns uninformative or empty results, the model can get trapped in repetitive loops, struggling to reformulate its search queries.
• Context Window Limits: Few-shot prompting with detailed trajectories consumes significant context space, limiting the number of in-context exemplars.