FrontierCode: top AI models still fail over 40% of the time at writing code maintainers would merge

Artificial Analysis has added FrontierCode 1.1, a coding benchmark from Cognition, to its leaderboards, publishing its first independent results on 12 October 2026. Each task asks an agent to make a change to a real open source repository and judges whether a maintainer would merge it, penalising things like oversized diffs, edits outside the relevant files and unhelpful tests, not just failing tests. Cognition built it with maintainers of 36 repositories, who spent more than 40 hours on each task. Artificial Analysis ran the 100-task main set, three attempts per task, with every model in the same agent harness. Leading models still fail more than 40% of the time. Claude Opus 5.5 (max with fallback) leads with an average score of 52% and a pass rate of 58%, followed by GPT-6.1 Sol (max) and Claude Fable 5.1 (max with fallback) at 49%. Nine models from six labs score between 43% and 52%, and MiMo-V2.6-Pro is the top open weights model at 45%. Cost varies widely: GPT-6.1 Sol (max) scored 49% at $0.81 per task, while Claude Opus 5.5 (max with fallback) cost $10.15 per task for 3 more points. Nine of the 100 tasks were passed by no model on any attempt.