No Autonomy Without Scalable Oversight
What to expect as we enter the Year of The Judge.
AI security researcher · Dreadnode
Full Contact
Alignment
I build scalable oversight and control systems for autonomous agents operating in adversarial, high-consequence environments.
Shane CaldwellResearcher & engineer
Latest research
July 2026 · arXiv
ScopeJudge: Cost-Aware Pre-Execution Gating for Offensive Security Agents
A benchmark of 4,897 offensive-security agent tool calls for evaluating how well runtime monitors detect out-of-scope actions as the amount of available context varies.
Notes from the work
Recent writing
ProofJudge: Can we align vibe-proving with human taste?
Towards measuring alignment with human taste in autoformalization with judge agents.
The Tests All Pass
METR’s SWE-bench analysis shows us taste isn’t verifiable.
Intro to GPUs For the Researcher
Getting comfortable with the hardware on a quest for more MFU.
Browse by thread