- Resolves
- June 30, 2027 at 11:59 PM UTC
- Evidence
- The official SWE-bench leaderboard and the lab's own published evaluation.
- Decision
- Count only scores on SWE-bench Verified reported by the benchmark maintainers or reproduced by them. The model must be usable by the public through an API or product on the resolution date.
- Void
- Void if the benchmark is retired or materially changed before the resolution date.
- Resolver
- Objective Source
Comments
No comments yet.
Sign in to comment