yeah chunking is where all the quality actually lives. everyone benchmarks embedding models and nobody benchmarks their splitter

I would push back a little on the framing, only because it makes segmentation sound like a solved step you just have to do carefully. It is not. A good split depends on what the downstream question looks like, and you almost never know that at ingest time. The version I have landed on is to keep the document structure alongside the chunks — page boundaries, heading depth, table extents — so you can re-split later without re-crawling anything. Treating chunking as a one-way ingest decision is the real mistake, not picking the wrong window size.

同意上面这位。保留结构确实关键。我们后来的做法是把页码、标题层级和表格边界都当元数据存下来，需要重切的时候直接从原始结构重来一遍，不用再爬一次，省了非常多时间。

对 我这次就是没存结构 想重切等于全部重来 下次一定 😅