Twenty Billion Tokens of What, Exactly?
Looking at the data and letting it look back at us.
Writing
Pretraining at home: 20B tokens from 222 hours to 12
Optimizing training a Llama 3.2 1B model so we can pretrain in a day without going broke.
Offsec Evals: Growing Up In The Dark Forest
If you contribute a public benchmark, are you giving free capability to your competitors?
DiLoCo: Data Parallelism for the Datacenter Poor
Distributed training sans datacenter.
RL Needed LLMs Because Agency Requires Priors
We tried RL once. It didn’t work. I’m confident it will this time.
GPT-5 is Good, Actually: The Agony and Ecstasy of Public Benchmarks
An attempt to explain why benchmarks are either bad or secret, and why the bar charts don’t matter so much.
The Religious Devotion of Haskell
A brief(ish) anecdote and investigation into the religious devotion of Haskell programmers.
The Input Sanitization Perspective on Prompt Injection
So, you mixed user input and instructions.
Infosec's Data Problem
Exploring the unique challenges of doing real science in the world’s most paranoid industry.
Deep Reinforcement Learning for Security: Toward an Autonomous Pentesting Agent
A manifesto on RL in cybersecurity, from when deep RL was the thing.