Research on Shane Caldwell https://hackbot.dad/tags/research/ Recent content in Research on Shane Caldwell Shane Caldwell https://hackbot.dad/ https://hackbot.dad/ Hugo -- 0.146.2 en-us Fri, 12 Dec 2025 00:00:00 +0000 All Reduce Across the Atlantic: Bandwidth in Decentralized Training https://hackbot.dad/writing/all-reduce-across-atlantic/ Fri, 12 Dec 2025 00:00:00 +0000 https://hackbot.dad/writing/all-reduce-across-atlantic/ The practical realities of devestatingly high communication cost in training. Pretraining at home: 20B tokens from 222 hours to 12 https://hackbot.dad/writing/pretraining-at-home/ Sun, 23 Nov 2025 00:00:00 +0000 https://hackbot.dad/writing/pretraining-at-home/ Optimizing training a Llama 3.2 1B model so we can pretrain in a day without going broke. RL Needed LLMs Because Agency Requires Priors https://hackbot.dad/writing/rl-llms-and-priors/ Tue, 19 Aug 2025 00:00:00 +0000 https://hackbot.dad/writing/rl-llms-and-priors/ We tried RL once. It didn't work. I'm confident it will this time. GPT-5 is Good, Actually: The Agony and Ecstasy of Public Benchmarks https://hackbot.dad/writing/agony-and-ecstasy-evals/ Sun, 17 Aug 2025 00:00:00 +0000 https://hackbot.dad/writing/agony-and-ecstasy-evals/ An attempt to explain why benchmarks are either bad or secret, and why the bar charts don't matter so much.