No Autonomy Without Scalable Oversight
What to expect as we enter the Year of The Judge.
Essays, field notes, and technical deep dives
Writing
Long-form work on autonomous agents, evaluation, machine learning systems, and offensive security.
2026
ProofJudge: Can we align vibe-proving with human taste?
Towards measuring alignment with human taste in autoformalization with judge agents.
The Tests All Pass
METR’s SWE-bench analysis shows us taste isn’t verifiable.
Intro to GPUs For the Researcher
Getting comfortable with the hardware on a quest for more MFU.
2025
All Reduce Across the Atlantic: Bandwidth in Decentralized Training
The practical realities of devestatingly high communication cost in training.
Twenty Billion Tokens of What, Exactly?
Looking at the data and letting it look back at us.
Pretraining at home: 20B tokens from 222 hours to 12
Optimizing training a Llama 3.2 1B model so we can pretrain in a day without going broke.
Offsec Evals: Growing Up In The Dark Forest
If you contribute a public benchmark, are you giving free capability to your competitors?
DiLoCo: Data Parallelism for the Datacenter Poor
Distributed training sans datacenter.
RL Needed LLMs Because Agency Requires Priors
We tried RL once. It didn’t work. I’m confident it will this time.
GPT-5 is Good, Actually: The Agony and Ecstasy of Public Benchmarks
An attempt to explain why benchmarks are either bad or secret, and why the bar charts don’t matter so much.