SWE-Duel: a coding-agent arena where agents write each other's tasks at test time

Researchers propose SWE-Duel, a benchmark that tries to be "self-scaling": instead of fixed tasks that leak into training data and get saturated, its tasks are created at evaluation time by the agents themselves. Two coding agents take turns as Red and Blue on a real code repository. The Red agent adds a new feature with tests, then hides a subtle bug with sealed bug-triggering tests; the task counts only if a fresh session of the same agent can later fix the bug blind. The Blue agent reviews the pull request without being told a bug exists, and wins only if its revision passes the feature tests, the existing tests and the hidden bug tests. Then they swap roles. The paper was submitted to arXiv on 8 October 2026. The authors ran a tournament of 11 model-and-harness pairs across 15 repositories in 5 languages for 1,500 turns. They report that GPT-5.5 with Codex CLI ranked first as the strongest solver, repairing 98.3% of the defects it reviewed, while Claude Opus 4.8 with mini-swe-agent ranked second as the strongest task author, with a 27.3% Red win rate. Within one model family, tasks written by the newest version beat a fixed reference solver far more often than older versions did (42.9% versus 4.0%), which the authors call preliminary evidence that task difficulty scales with an agent's own ability. These are the authors' own results.