SWE-bench
SWE-bench: Can Language Models Resolve Real-world Github Issues?
SWE-bench scores 4.6/10 (Emerging) in Benchmarks & Resources.
Kind Resource Language Python License MIT Stars 5.3k★ Repo swe-bench/swe-bench
Quality
Emerging 4.6
Adoption 43
Activity 11
Maturity 87
Community 49
Capability 50
Show the math
Overall 4.6/10
value = 4.6/10
score = value
Adoption 4.9/10
value = 43.2/100
score = 1 + 9 * value / 100
Activity 2/10
value = 11/100
score = 1 + 9 * value / 100
Maturity 8.8/10
value = 86.7/100
score = 1 + 9 * value / 100
Community 5.4/10
value = 49.4/100
score = 1 + 9 * value / 100
Capability 5.5/10
value = 50/100
score = 1 + 9 * value / 100
Key metrics
5.3k★ GitHub stars
0/90d Recent commits
0 devs/90d Recent contributors
2.7y Project age
3mo ago Last commit
Gotchas
No gotchas documented yet. Contribute one if you know a constraint we missed.
benchmark coding evaluation agents