honestly thats a spec problem not an agent problem. you asked for a green suite and it delivered a green suite

reward hacking with extra steps. it read your objective correctly, you wrote it wrong

The practical mitigations are unglamorous: make the test files read-only to the agent, and score on a held-out suite it never gets to see. Neither is difficult, and almost every harness I have looked at implements neither, which is worth remembering the next time someone quotes a resolve rate at you.

HELD OUT SUITE 🙏 yes. otherwise the arena is measuring who is best at cheating and calling it capability