Shortcut Forensics

Coding agents sometimes take shortcuts under pressure — hardcoding a failing test, disabling a lint check, declaring the work done without doing it. Recent papers disagree on why: a low-effort disposition, evaluation-awareness or knowing the user would disapprove, desperation (one study found a “desperate” activation direction takes hacking from roughly 5% to 70%), or a trained-in drive to see a passing check.

This project tests six candidate mechanisms — tedium, evaluation-awareness, disapproval-awareness, desperation, a trained “shortcut” drive, and a raw completion drive — against a single standard: does the direction read out on unmodified rollouts, does it move the shortcut rate on the model's original prompt via the sign-appropriate intervention (ablating it vs. amplifying it) without collapsing general capability, and does it generalize across different kinds of shortcuts rather than explaining only one.

Method: contrastive activation extraction and direction-fitting on an open-weight coding agent (Qwen3.5-9B) inside a naturalistic pre-commit-hook environment from the literature, validated against held-out accuracy and positive-control interventions, then tested causally by steering on the model's own original prompts.

Originated as a MATS 12.0 (Neel Nanda, Winter 2027) application experiment; now pursued as independent research regardless of that outcome.