← Flash PapersAligning AI: Inside OpenAI's InstructGPT & RLHF Pipeline

Share this Flash Paper

Aligning AI: Inside OpenAI's InstructGPT & RLHF Pipeline

https://flashpapers.ai/p/xquudxqh6k
Post to XPost to BlueskyShare on LinkedInSubmit to Hacker NewsSubmit to RedditEmail a link
Make your own
Long Ouyang, Jeff Wu +18 more
cs.CL2022arXiv:2203.02155Code· 1.3k★· updated 4y ago
Details ▸
Human Preference Winrate vs GPT-3 175B:85 ± 3%· API prompt distributionHallucination Rate (Closed Domain):21%· API prompt distributionToxicity Reduction (Respectful Prompt):25% fewer toxic outputs· RealToxicityPromptsCompute Cost (175B SFT):4.9 petaflops/s-daysCompute Cost (175B PPO-ptx):60 petaflops/s-days
Concepts:Proximal Policy Optimization