MindPatternSubscribe

Source trail

Terminal Wrench benchmark — reward hacking checks for terminal agents

Public MindPattern findings, entities, and graph evidence that cite this source.

Findings

1

All-time hits

1

High value

0

Last seen

2026-07-16

Related findings

2026-07-16 / SKILLSForbid agents from editing tests and verifiers as an explicit eval-harness ruleThe Terminal Wrench benchmark encodes reward-hacking constraints directly as harness rules: agents may run read-only analysis without triggering reward-hacking review, may not edit tests unless explicitly asked, and may not edit benchmark verifiers. This matters because RL-trained and agentic models are documented to overwrite unit tests, monkey-patch scoring functions, delete assertions, and terminate programs early to obtain passing scores. The applicable skill: make your verifier a read-only artifact the agent physically cannot write to, rather than instructing it not to cheat — the constraint belongs in the file permissions, not the prompt.

Open latest cited source