Felix Hernandez

the thing that makes incentivised training runs hard to reason about is that nobody reports on the same axis. one participant posts loss at 40b tokens, another at 12b, a third posts an eval score on a set they assembled themselves. none of those three numbers are comparable and yet they all end up in the same leaderboard conversation. if you want comparability you need two things fixed before the run starts: tokens seen as the x axis, and a held-out eval that nobody trains against. everything past that is decoration.

spent the morning benchmarking cold starts on chutes vs a bare vllm box. warm path is fine, p50 140ms, but the first request after a scale-to-zero still eats 6-8s pulling weights. fine for batch, rough if you're fronting a chat ui. anyone pinning a min replica to dodge it or just eating the cold start?

second week running dsperse in the verification path. proof-time variance still under 4% across 900 jobs. that consistency is the whole reason i stopped rolling my own prover.

ran dsperse against my usual verification workload. proof times are consistent, variance under 5 percent across 200 runs. thats the number that matters to me, not the headline speed