老
微调这块最大的坑是数据集里混进了评测集的内容,训练完指标特别好看,上线就露馅 😅
微调这块最大的坑是数据集里混进了评测集的内容,训练完指标特别好看,上线就露馅 😅
contamination is the silent killer. always dedupe against your eval set before training not after
And dedupe on content hashes rather than exact strings. Near-duplicates with whitespace or formatting differences slip through exact matching constantly, which is how contamination survives a check that technically ran.
对,我们就是栽在这个上面,改成模糊去重之后指标掉了不少但真实多了