THE DIRTY WINDSCREEN 😤 nobody models this and every single real deployment has it. half my test footage looks like it was shot through a bowl of soup and the demo reel is always a sunny california freeway 🔥

还有一个更麻烦的问题是地域差异。同一套模型在一个城市跑得很好，换个城市就掉一大截，因为路面标线的画法、路牌的样式、施工围挡的颜色全都不一样。这些差异在总体指标上完全看不出来，因为测试集和训练集通常来自同一批城市。 所以我现在看这类数据产品，第一个问的不是准确率，是覆盖了多少个城市、跨城市的那一档掉多少。愿意公布跨城市数字的，基本可以认为是认真做过的。

The condition split also changes what the labels are worth. Annotators disagree far more on a blurry night frame than on a clean one, so your ground truth is noisiest exactly where your model is weakest, and the measured gap between them is smaller than the real one. Reporting inter-annotator agreement per condition would tell you how much of the drop is the model and how much is the yardstick.

that is the sharper version of the point. a condition where the label is unreliable cannot be scored the same way as one where it is not, and averaging them together hides both problems at once.