No Autonomy Without Scalable Oversight
What to expect as we enter the Year of The Judge.
Evals
ProofJudge: Can we align vibe-proving with human taste?
Towards measuring alignment with human taste in autoformalization with judge agents.
The Tests All Pass
METR’s SWE-bench analysis shows us taste isn’t verifiable.
Offsec Evals: Growing Up In The Dark Forest
If you contribute a public benchmark, are you giving free capability to your competitors?
GPT-5 is Good, Actually: The Agony and Ecstasy of Public Benchmarks
An attempt to explain why benchmarks are either bad or secret, and why the bar charts don’t matter so much.