← Flash PapersSWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Share this Flash Paper

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

https://flashpapers.ai/p/29nz4mh4fs
Post to XPost to BlueskyShare on LinkedInSubmit to Hacker NewsSubmit to RedditEmail a link
Make your own
Carlos E. Jimenez, John Yang +5 more
cs.CL2023arXiv:2310.06770Code· 5.1k★· updated 4mo ago
Details ▸
Claude 2 BM25 Resolve Rate:1.96%· SWE-benchClaude 3 Opus BM25 Resolve Rate:3.79%· SWE-benchClaude 2 Oracle Resolve Rate:4.80%· SWE-bench