been poking at trajectoryrl for a few days. its an RL subnet, rewards miners for producing good trajectories on a set of tasks. the idea is solid and the demo runs. what im neutral on is whether the task set is broad enough to matter or if its narrow benchmarks that look good but dont generalize. anyone gone deeper
was gonna say this looks mid 😬 but i ran a couple custom tasks through it and it held up better than i expected 👀 ok im warmer on it now, not a shill just surprised 🤷
One thing to flag for anyone evaluating RL subnets generally. A demo running on the teams own task set tells you almost nothing about generalization. The question to ask TrajectoryRL is whether the held out tasks are disclosed and whether miners can see them before scoring. If the eval set is gameable the whole reward signal is. Not saying it is here, just that thats the load bearing question.
i was ready to write this off as another benchmark farm. but the task rotation is actually nontrivial, they swap the set often enough that pure overfitting doesnt pay. fine, ill upgrade it from junk to worth watching
the demo runs is doing heavy lifting here. rl demos always run on the tasks they were tuned for. send it something off distribution and watch it fall apart. id be skeptical till someone tests it cold
本来觉得就是个刷榜的 看了下他们任务集 居然有几个挺难泛化的 行 我态度软化一点 不全是花架子
narrow benchmarks that look good is the default state of every rl subnet. assume that until proven otherwise
whats the actual compute cost to compete as a miner here. rl trajectories arent cheap. if the reward doesnt cover the gpu hours nobodys gonna stick around past the farm
