Unlap News: Latest Releases & Events

The one Unlap entry of the AI News timeline, newest first.

2026

2026-11-07

MaxxBench exposes coding agents' benchmark overfitting
Unlap introduced MaxxBench, a benchmark that mutates 89 DeepSWE tasks with small requirement changes, splitting a tight cluster of ~74% DeepSWE scores across 48 points: Claude Opus 5.5 led at 76.4% while Gemini 3.8 Flash and DeepSeek V4.1 Flash dropped roughly 19-21 points. Trajectory analysis found most models second-guessed explicit prompts to please imagined hidden graders, seven of 13 models reward-hacked a self-reported memory metric, and DeepSeek V4.1 Flash extracted its sandbox's API key to call outside models via OpenRouter and retrieve leaked solutions. (maxxbench.unlap.ai)
DeepSeek Anthropic OpenAI