Loagaeth
e26fe1c3f5
feat(kb): add Markdown-aware chunker for structured documents (#8151)
* feat(kb): add Markdown-aware chunker for structured documents
* fix: address review feedback from sourcery-ai and gemini
- Clamp max_heading_depth to 1-6 to prevent regex errors
- Deduct prefix length when splitting oversized sections
- Replace hardcoded "[续]" with configurable continuation_prefix
- Skip fenced code blocks when detecting headings
- Cap pending size to prevent chunks exceeding chunk_size
- Refactor into dataclass + helper methods
* fix: handle unmatched fenced code block at EOF
* fix: prevent chunks exceeding chunk_size with long heading prefixes
* chore: rf
---------
Co-authored-by: Soulter <905617992@qq.com>
2026-05-30 17:43:13 +08:00
..
2026-05-30 17:16:36 +08:00
2026-05-05 01:05:43 +08:00
2026-05-13 14:38:11 +08:00
2026-05-30 17:16:36 +08:00
2026-05-03 14:49:05 +08:00
2026-05-26 09:37:01 +09:00
2026-05-30 17:43:13 +08:00
2026-05-28 08:20:11 +08:00
2026-05-30 17:16:36 +08:00
2026-05-29 21:06:04 +08:00
2026-05-30 17:26:36 +08:00
2026-05-03 16:37:36 +08:00
2026-05-15 17:00:03 +08:00
2026-05-30 14:01:25 +08:00
2026-05-22 19:11:38 +08:00
2026-03-11 14:21:55 +09:00
2026-04-14 18:55:05 +08:00
2026-04-23 17:46:53 +08:00
2026-04-28 01:40:14 +09:00
2026-05-03 14:37:37 +08:00
2026-05-30 17:16:36 +08:00
2026-03-01 16:10:35 +09:00
2026-04-29 11:05:38 +08:00
2026-03-01 00:23:47 +08:00
2026-04-14 16:27:06 +08:00
2026-03-03 12:58:59 +08:00
2026-02-28 13:34:12 +09:00
2026-05-05 21:16:53 +08:00
2026-04-22 11:51:12 +08:00
2026-02-28 13:34:12 +09:00
2026-03-17 15:42:15 +08:00
2026-03-16 12:34:21 +08:00
2026-05-14 00:40:24 +08:00
2026-05-14 00:40:24 +08:00