No Autonomy Without Scalable Oversight
What to expect as we enter the Year of The Judge.
Llms
ProofJudge: Can we align vibe-proving with human taste?
Towards measuring alignment with human taste in autoformalization with judge agents.
The Tests All Pass
METR’s SWE-bench analysis shows us taste isn’t verifiable.
Intro to GPUs For the Researcher
Getting comfortable with the hardware on a quest for more MFU.
All Reduce Across the Atlantic: Bandwidth in Decentralized Training
The practical realities of devestatingly high communication cost in training.
Twenty Billion Tokens of What, Exactly?
Looking at the data and letting it look back at us.
Pretraining at home: 20B tokens from 222 hours to 12
Optimizing training a Llama 3.2 1B model so we can pretrain in a day without going broke.
Offsec Evals: Growing Up In The Dark Forest
If you contribute a public benchmark, are you giving free capability to your competitors?
DiLoCo: Data Parallelism for the Datacenter Poor
Distributed training sans datacenter.
RL Needed LLMs Because Agency Requires Priors
We tried RL once. It didn’t work. I’m confident it will this time.