千岚之夏
bec0de2e2b
fix: defer faiss C library import to prevent process hang on startup ( #8696 )
...
* fix: defer faiss C library import to prevent process hang on startup
Move top-level `import faiss` from embedding_storage.py to __init__() method,
and make the EmbeddingStorage import in vec_db.py lazy. This prevents the faiss
C library from being loaded at module import time, which can cause process hang
(SIGILL or deadlock) with faiss-cpu 1.14.2 on certain CPU architectures.
Ref #8695
* chore: apply review suggestions
- Preserve original exception context with `from e` when re-raising ImportError
- Move EmbeddingStorage import back to top level in vec_db.py (safe now that
faiss is no longer imported at module level in embedding_storage.py)
* ci: trigger re-run
---------
Co-authored-by: Blueteemo <Blueteemo@users.noreply.github.com >
2026-06-10 15:31:48 +08:00
Weilong Liao
c70a1924fe
Revert "fix SQLAlchemy compatibility issues on macOS" ( #8638 )
...
* Revert "fix SQLAlchemy compatibility issues on macOS (#7724 )"
This reverts commit 2d78626840
2026-06-07 10:50:33 +08:00
hibiki233i
2d78626840
fix SQLAlchemy compatibility issues on macOS ( #7724 )
...
* Stabilize packaged SQLite knowledge base initialization
* Apply suggestion from @sourcery-ai[bot]
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Apply suggestion from @gemini-code-assist[bot]
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix: updating database URL handling and ensuring unique document IDs
* fix: preserve sqlite pragmas with null pool
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: 邹永赫 <1259085392@qq.com >
2026-05-26 09:37:01 +09:00
C₂₂H₂₅NO₆
d9ab35348e
fix: drop legacy documents_fts table if exists ( #7706 )
...
* fix: recover FTS5 index from legacy documents_fts table
* fix: normalize SQL whitespace when checking contentless_delete
2026-04-21 22:27:40 +08:00
Soulter
352455197d
feat: implement FTS5 support in knowledge base sparse retrieving stage ( #7648 )
...
* feat: implement FTS5 support in DocumentStorage and SparseRetriever with tokenizer enhancements
* feat: optimize FTS row handling in DocumentStorage and update query tokenization in SparseRetriever
2026-04-18 19:57:27 +08:00
Gargantua
cc72c01c0e
fix: improve knowledge base upload error messages ( #7534 )
...
* fix: improve knowledge base upload error messages
* fix: deduplicate knowledge base upload logs
* fix: handle type errors in kb embedding validation
2026-04-14 16:27:06 +08:00
Soulter
52beeef83c
feat(skill_manager): normalize and rename legacy skill markdown files to SKILL.md ( #6757 )
...
* feat(skill_manager): normalize and rename legacy skill markdown files to `SKILL.md`
* fix(vec_db): format debug log message for empty batch insert
2026-03-22 00:43:30 +08:00
Stable Genius
589776ab3f
fix: skip empty knowledge-base embedding batches ( #6106 )
...
Co-authored-by: stablegenius49 <185121704+stablegenius49@users.noreply.github.com >
2026-03-21 20:56:13 +08:00
Dt8333
7dd95d8a59
chore: auto ann fix by ruff ( #4903 )
...
* chore: auto fix by ruff
* refactor: 统一修正返回类型注解为 None/bool 以匹配实现
* refactor: 将 _get_next_page 改为异步并移除多余的请求错误抛出
* refactor: 将 get_client 的返回类型改为 object
* style: 为 LarkMessageEvent 的相关方法添加返回类型注解 None
---------
Co-authored-by: Soulter <37870767+Soulter@users.noreply.github.com >
2026-02-09 00:22:24 +08:00
Dt8333
f624971613
chore: fix bunches of type checking errors ( #3213 )
...
* chore(core.utils): 🚨 修正错误Lint
* chore(core.provider): 🚨 修复基类错误Lint
* chore(core.utils): 补全session_get()的重载
* chore(core.provider): 🚨 修正实现错误Lint
* chore(core.platform): 🚨 修正platform基类和webchat的错误Lint
* chore(core.platform): 修正错误实现Lint
* fix(core.provider): 修复循环调用和错误assert
* chore(core.platform): 修复部分实现Lint
* chore(core.provider): 补充Dify.text_chat_stream的参数类型
* chore(core.pipeline): 🚨 修复错误Lint
* fix(core.slack): 补充遗漏导入
* chore(core.utils): 修复错误的session_get声明
* chore(core.platform): 移除Lark adapter import中的wildcard
* chore(core.db): 修复声明和部分逻辑
* chore(core.db): 添加typings，使faiss参数能被正确识别。
* chore(core): 修复声明
* chore(core): 修改声明
* chore: 补充faiss声明
* chore(dashboard): 修改实现，减少报错
* chore(package): 修改部分声明与实现，减少报错
* chore(core): 添加Handler的overload，以去除部分assert同时通过类型检查
* chore(core.pipeline): 修改Pipeline Scheduler的execute，将判断属性改为判断类型，通过静态类型检查
* chore(core.config): 添加类型标注，通过类型检查
* chore(core.message): 为File._download_file添加检查，通过类型检查
* fix: 将断言改为条件判断以实现优雅关闭的容错性
* refactor: 移除 discord 客户端中的 assert，改用 if None 判断并抛出异常
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: DiscordPlatformAdapter 对 self.client.user 为 None 做日志并返回，移除断言
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: 增强 Lark 相关空值/异常检查并完善日志输出
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* refactor: 将断言替换为条件检查并加入日志与错误处理
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* chore: 移除LLM生成的无用注释
* refactor: 使用 File.get_file 替换下载逻辑并移除 assert，提供默认 filename
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: Slack Socket 未初始化抛出运行时异常，图片 URL 判空改为非空判断
* refactor: 将 WeChatPadProAdapter 的断言改为空值判断并添加日志
* refactor: 使用 isinstance 替代断言实现类型判断，便于静态检查
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: 去除cast，直接使用字段与字典访问，修正端口解析
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* refactor: 使用 match-case 重构 ProviderManager 加载并通过类型检查抛出 TypeError
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: group_name_display 时若 group 对象为空则记录错误并返回
* fix: 将 _get_current_persona_id 的 assert 替换成 if guard 并返回 None
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: 优化插件目录存在性检查及图片URL非空验证，更新JSON排序配置
* fix: 将 datetime_str 的 assert 替换为显式检查并抛出异常
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* refactor: 移除 cast，改为运行时检查并在找不到调度器时跳过
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* refactor: 移除 cast，改用 isinstance 检查 FaissVecDB 并警告
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: 删除 typing.cast 导入，并在获取文件绝对路径前校验 file_
* refactor: 移除 typing.cast，简化内容安全检查调用
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* refactor: 将 PlatformMetadata.id 设为必填并在注册时传入 id，移除 cast
* refactor: 移除 cast，改用 HasInitialize 与 isinstance 进行初始化
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: 为 ProviderManager.initialize 增加ID类型判断，避免 None 导致 get 失败
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* refactor: 为 OTTSProvider 与 AzureNativeProvider 引入 _client 与 client 属性改进上下文管理
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: 为 Whisper 自托管源添加模型未初始化校验并直接调用 transcribe
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* refactor: 移除未使用的 cast 导入并简化 platform_name 赋值
* refactor: 引入 cast 并对 id 使用 cast(str, ...) 提升类型安全
* fix: 将 _id_to_sid 返回改为 str，空值返回空串；对 id 与 message_id 使用 cast
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* refactor: 重构 Discord 处理逻辑：强制 类型转换、优先斜杠指令并优化提及判断
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
* fix: 统一对 id 获取执行 cast，并在微信消息解析失败时抛错
* Revert "fix: 去除cast，直接使用字段与字典访问，修正端口解析"
This reverts commit 1cbfdf9d1baider@aider.chat >
* refactor: 调整 call_handler 泛型、Discord 通道注解及 FishAudioTTS API 请求类型
* refactor: 使用 col(...) 替代列引用并对结果进行 CursorResult 强转
* chore: ruff format
---------
Co-authored-by: aider (openai/gemini-3-pro-high) <aider@aider.chat >
Co-authored-by: Soulter <905617992@qq.com >
2025-12-09 14:13:47 +08:00
LIghtJUNction
0b7fc29ac4
style: add ruff lint module of isort and pyupgrade, and some ruff check fix ( #3214 )
...
Co-authored-by: Dt8333 <25431943+Dt8333@users.noreply.github.com >
Co-authored-by: Soulter <905617992@qq.com >
2025-11-01 13:26:19 +08:00
lxfight
594ccff9c8
fix: 添加数据库连接检查和知识库终止功能，增强错误处理和清理逻辑，修复知识库无法删除的问题
2025-10-25 11:56:37 +08:00
Soulter
8f021eb35a
feat: refactor document storage to use SQLModel and enhance database operations
2025-10-24 23:17:37 +08:00
Soulter
9b004f3d2f
feat: update document retrieval to include limit and offset parameters
2025-10-24 17:38:22 +08:00
Soulter
9430e3090d
feat: add progress callback for document upload and enhance upload progress tracking
2025-10-24 17:13:44 +08:00
Soulter
ba44f9117b
feat: enhance document upload process with batch settings and improved chunk handling
2025-10-24 16:37:37 +08:00
Soulter
2f130ba009
feat: delete chunk and delete document
2025-10-24 13:59:17 +08:00
Soulter
7e0a50fbf2
feat: enhance knowledge base retrieval with chunk metadata and pagination support; remove unused chunk model
2025-10-24 00:44:40 +08:00
Soulter
e3aa1315ae
stage
2025-10-23 00:31:15 +08:00
Soulter
7740e1e131
ci: add ci stage of code format checking ( #2750 )
...
* style: ruff format
* ci(dashboard-ci): ensure GitHub Release action only runs on push events
* ci(code-format): ruff format and ruff check
2025-09-14 18:05:58 +08:00
Soulter
1a84ebbb1e
fix: remove debug print statement for reranked results in FaissVecDB
2025-08-19 17:57:16 +08:00
Soulter
c9c0352369
feat: 知识库支持配置重排序模型
2025-08-19 17:51:01 +08:00
Soulter
2ab5810f13
perf: improve transaction performance in vector db
2025-05-30 12:59:26 +08:00
Soulter
af934c5d09
fix: correct dimension typo and enhance API registration logic
2025-05-30 11:42:39 +08:00
Soulter
a36e11973d
perf: code quality
...
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
2025-05-23 16:56:09 +08:00
Soulter
5125568ea2
perf: 交换 if/else 表达式的分支以删除否定
...
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
2025-05-23 16:49:08 +08:00
Soulter
4107a17370
chore: add faiss and aiosqlite deps
2025-05-23 14:04:13 +08:00
Soulter
06b4d8f169
perf: vecdb similarity type
2025-05-23 13:45:00 +08:00
Soulter
1c0c820746
remove: loguru
2025-05-23 13:42:17 +08:00
Soulter
d061403a28
remove: loguru
2025-05-23 13:39:20 +08:00
Soulter
5c092321a6
feat: faiss vecdb implementation
...
remove: old knowledgedb deps
2025-05-23 13:16:24 +08:00
