新增预设工作台：编辑、版本树、结构检查、用例试跑放在同一页。
改预设要能一边写一边看装出来的 messages，还要多模型对照，所以把编辑与试跑拆成两个视图塞进一个页面；版本存完整快照、可分叉，用例全局共用一份、运行记录按预设存。顺带让读预设文件先剥 BOM，静态资源改成不缓存，免得改了脚本还要强刷。
This commit is contained in:
@@ -10,6 +10,7 @@
|
||||
- **游玩**：在钉住的规格下回合推进；可切换呈现壳，保存游玩进度
|
||||
- **技能包**：能力以磁盘上的技能包装载（当前主包为「世界模拟器」相关配方与模块）（目前不需要手动选择）
|
||||
- **预设与库**：兼容 SillyTavern 风格的提示词预设；文风库 / 偏好库可注入创作节点，种子可从仓库恢复
|
||||
- **预设工作台**：`/presets.html` 内编辑模块与生成参数、版本树分叉、结构检查、用例组与多模型试跑（见 [preset-workbench.md](docs/preset-workbench.md)）
|
||||
- **多接口**：分组管理 API、嗅探型号、切换同一接口下的模型
|
||||
- **本机优先**：Web（`localhost:23337`）或 `start-app.bat` 桌面壳；运维可用 `console.bat`
|
||||
|
||||
@@ -105,6 +106,7 @@ Windows 运维脚本：`console.bat`（查询 / 启停 / 日志）。入口实
|
||||
| [run-snapshot.md](docs/run-snapshot.md) | 存档 / 快照（以代码现状为准） |
|
||||
| [play-presentation-shells.md](docs/play-presentation-shells.md) | 游玩呈现壳 |
|
||||
| [preset-format.md](docs/preset-format.md) | 预设格式 |
|
||||
| [preset-workbench.md](docs/preset-workbench.md) | 预设工作台（编辑 / 试跑） |
|
||||
| [skill-format.md](docs/skill-format.md) 等 | 技能 / 编排器 / Worker 格式 |
|
||||
| [px-roadmap.md](docs/px-roadmap.md) | 产品路线图 |
|
||||
| [skills/README.md](skills/README.md) | 技能包目录说明 |
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 预设格式
|
||||
|
||||
> 编辑、版本树、用例组与多模型试跑见 [preset-workbench.md](preset-workbench.md)。
|
||||
|
||||
## 1. 定位
|
||||
|
||||
Preset 是一次 LLM 请求的上下文编排资产。它负责描述 prompt manager 中有哪些条目、这些条目的启用状态与排列顺序，以及本次生成使用哪些模型参数。
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
# 预设工作台
|
||||
|
||||
本仓库自用的预设编辑 / 试跑页面：`/presets.html`。它只编辑本仓库的原生 `PresetPackage`，
|
||||
不做 SillyTavern 兼容导入（导入仍在设置页）；目标是把「改一版 → 看结构 → 看实际 messages →
|
||||
多模型试跑 → 标记好坏」压在同一页里。
|
||||
|
||||
## 1. 页面分工
|
||||
|
||||
页面是「一条身份条 + 两个视图 + 两个抽屉」。编辑与试跑是两件不同任务：编辑要连续写，试跑要宽度做对照，
|
||||
因此不挤在同一个视口里。
|
||||
|
||||
```text
|
||||
身份条 预设选择 / 新建 / 改名 / 脏状态 / 重新载入 / 试跑入口 / 版本树入口 / 保存两档 / 应用到对话页
|
||||
唯一脏状态、唯一 primary（应用到对话页）都在这里，任何视图都能看到。
|
||||
保存两档：「保存至当前版本」覆盖当前节点，「保存为新版本」追加子节点。
|
||||
版本 id 不上条（只做 title），要看序列去版本树。
|
||||
|
||||
编辑视图 左轨（240）发送顺序大纲：序号 1..N、启用开关、位置标记、错误红点；
|
||||
行可直接拖动改顺序（落点画一条强调色横线），标记洞也能拖——左轨顺序就是装配顺序；
|
||||
未加入顺序的模块折叠在底部；最后两栏是「生成参数」与「假历史」。
|
||||
中区 当前模块正文（一次只编辑一个；标记洞只显示说明，不能改名/删除，但能移顺序）。
|
||||
「假历史」中区是逐条编辑：每条一个角色下拉 + 一整句正文 + 加一句 / 删除 / 上下移。
|
||||
右栏（380）实际发出的 messages：顶部是结构检查状态带 + 用例选择 + 随时重算。
|
||||
改正文、切模块、切用例都会本地重算预览（不调模型）。
|
||||
|
||||
试跑视图 顶部运行栏一行：当前用例名 · 模型（下拉多选：勾选不关菜单，可全选可用 / 清空）
|
||||
· 已选条（只占一行，超出可左右拖，点一项取消） · 「运行此用例」。
|
||||
左侧用例列表（一条 = 一个用例，点开进大浮窗编辑），右侧全宽结果网格（同一用例 × 多模型对照）。
|
||||
低频入口：身份条上的「试跑 · 上次结果摘要」按钮，进入后按钮变成「← 编辑」。
|
||||
|
||||
抽屉 版本是右侧抽屉（版本树 / 说明 / 保存 / 载入 / 分叉）；
|
||||
用例表单是**大浮窗**（左列 5 个字段，右列历史条目编辑器），字段多、历史要宽度，
|
||||
不再挤在窄侧栏里。两者都是低频或临时的，不常驻占内容宽度。
|
||||
```
|
||||
|
||||
编辑视图一屏不滚：大纲、正文、预览各自滚动。预览在 ≤1180px 退成右侧浮层，≤860px 大纲也退成浮层。
|
||||
|
||||
## 2. 草稿、版本与应用
|
||||
|
||||
```text
|
||||
草稿 页面内存里的 state.preset；每次编辑只改这里。
|
||||
重新载入 丢掉草稿，从 presets/<presetId>.json 重读；不动版本树。
|
||||
保存至当前版本 覆盖当前节点（amend），不新增节点。
|
||||
保存为新版本 把草稿作为子节点追加到版本树，当前节点移到新节点。
|
||||
载入选中 把选中节点的完整快照载入草稿，并把当前节点切过去。
|
||||
从选中分叉 用当前草稿在选中节点下建子节点，原分支不动。
|
||||
应用到对话页 先保存为新版本，再写 presets/<presetId>.json；对话页从这里读。
|
||||
```
|
||||
|
||||
版本节点保存的是**完整 PresetPackage 快照**，不是 diff；因此任意节点都可独立载入。
|
||||
版本树文件：`<data>/presets/versions/<presetId>.json`。新建预设会同时落预设文件和根版本。
|
||||
|
||||
版本抽屉里是 git 式的树：竖线是分支，圆点是节点（实心 = 当前编辑位置），分叉另开一条线。
|
||||
点任一节点弹菜单：「读取」把这个快照换进编辑器，「从它分叉」用当前草稿在它下面开新分支，
|
||||
再往右是「复制 id」。底部四个按钮（保存为新版本 / 保存至当前版本 / 载入选中 / 从选中分叉）保留给键盘流。
|
||||
|
||||
两种保存别搞混：
|
||||
|
||||
```text
|
||||
保存为新版本（Ctrl+S） 在当前位置下追加子节点；历史留着，能回滚。
|
||||
保存至当前版本（Ctrl+Shift+S） 直接覆盖当前节点，不产生新节点。适合“这条还没定，先改它”。
|
||||
应用到对话页 先保存为新版本，再覆盖 presets/<presetId>.json。对话页读的是这个文件。
|
||||
```
|
||||
|
||||
装配只看 `prompt_order.enabled`；`prompts[].enabled` 是酒馆遗留字段。工作台的模块开关
|
||||
写的是顺序开关，因此和真实发送一致。
|
||||
|
||||
### 2.1 左轨定死顺序
|
||||
|
||||
```text
|
||||
左轨顺序 = 装配顺序 = 真实发送顺序。
|
||||
|
||||
装配（src/preset/markers.ts: applyExtendedMarkers）只补缺不重排：
|
||||
缺哪个洞就按锚点插进来（历史前 → 对话历史 → 历史后 → 本轮对话 → … → 本轮之后），
|
||||
已经在 prompt_order 里的条目一个不动，作者拖到哪儿就发在哪儿。
|
||||
|
||||
右侧预览只是展示：同一条装配路径 + 预设的「消息整形」。整形会合并消息（`strict` 默认），
|
||||
条数变少时预览元信息会写「整形 strict（装配 N 条 → 发出 M 条）」；
|
||||
想逐条原样发，把「生成参数 → 消息整形」改成 `none`。
|
||||
```
|
||||
|
||||
## 3. 结构检查
|
||||
|
||||
不调用模型，只查会在装配 / 发送时出问题的结构：
|
||||
|
||||
- 顺序引用了不存在的模块、同一模块在顺序里重复、顺序号重复；
|
||||
- 未启用 `chatHistory`、模块为空、全角 `｛｛`；
|
||||
- 未闭合的 `{{`（`{{{...}}` 的首个 `{` 是字面量，不算未闭合）；
|
||||
- 未识别宏（会原样外发）；
|
||||
- 标记位置重复启用。
|
||||
|
||||
`error` 只阻止「应用到对话页」，不阻止保存版本 / 预览；需要时可以强制写回。
|
||||
|
||||
## 4. 用例
|
||||
|
||||
一个用例就是一条记录：自己存、自己运行。页面上不出现「组」这个层级。
|
||||
|
||||
**用例是全局的，不和预设绑定**：一个用例只描述「要测的输入 + 上下文」，任何预设都能拿它去跑。
|
||||
盘上是 `<data>/presets/tests/global/*.json`（沿用旧 `TestCaseGroup` 形状，每份用例一个只装它的组文件）；
|
||||
旧的按预设分目录存的（`tests/<presetId>/*.json`）会在第一次访问时并进 global，原文件不删。
|
||||
删掉最后一份用例时组文件一并收掉。
|
||||
|
||||
「这个预设上次打开的是哪条用例」另外存：`<data>/presets/lab-state/<presetId>.json`，
|
||||
记 `selectedCaseId`（运行视图当前用例）与 `previewCaseId`（右栏预览用例）。
|
||||
写盘而不是塞浏览器缓存 —— 换浏览器、重开页面都还是同一条；切回别的预设又回到它自己那条。
|
||||
|
||||
用例列表的点击分工：**点卡片 = 选中（设为当前用例）**，只有卡片右边的「编辑」才打开用例编辑浮窗。
|
||||
|
||||
一个用例只包含变动项：
|
||||
|
||||
```text
|
||||
用例名 / 角色名（{{char}}） / 角色描述 / 世界书文本 / 用户输入 / 可选历史
|
||||
```
|
||||
|
||||
历史是**逐条编辑**的：每条一个角色下拉（`user` / `assistant`）加一整句正文，正文里的换行就是真换行，
|
||||
不再用「每行 `user: 内容`」的文本行和 `\n` 转义（旧写法会把不带头部的续行吞掉）。
|
||||
落盘仍是 `{role, content}[]`，装配时一条一个 message，接在预设假历史之后。
|
||||
组装走线上同一条路径：`assemblePlayWorkerMessages` → `shapeFramedMessages`。
|
||||
|
||||
## 4.1 假历史与示例对话是两回事
|
||||
|
||||
```text
|
||||
假历史 预设级、真多轮（user / assistant），进「对话.历史」区，排在真实历史之前。
|
||||
模型会把它当成已经说过的话，会占轮次；编辑器里正文换行原样保留。
|
||||
存储：PresetPackage.pseudoHistory，装配见 src/preset/pseudo-history.ts。
|
||||
|
||||
示例对话 酒馆 mes_example（<START> 分块），发给模型时是 system + example_user /
|
||||
example_assistant，不占轮次，只做 few-shot；正文仍是整段文本，保持酒馆兼容。
|
||||
```
|
||||
|
||||
两者互不覆盖：要「让模型以为前情已经发生」用假历史，要「示范写法但不冒充轮次」用示例对话。
|
||||
三者（预设假历史 / 用例历史 / 直聊种子历史）用的是同一个控件：`web/history-editor.js` + `web/history-editor.css`。
|
||||
每行左侧的 `⠿` 是拖动手柄：按住拖到另一行的上半 / 下半即插到它前面 / 后面，松手就到位；
|
||||
↑↓ 按钮保留（精确微调 / 不用鼠标拖时）。
|
||||
|
||||
## 5. 预览与多模型运行
|
||||
|
||||
```text
|
||||
预览 不调用模型；显示当前草稿 + 当前用例实际发出的 messages。
|
||||
运行 每个 用例 × 模型 发一次；默认最多 4 路并发、单次 180 秒超时。
|
||||
运行使用请求里的草稿，所以不必先保存版本。
|
||||
```
|
||||
|
||||
运行记录：`<data>/presets/runs/<presetId>/<runId>.json`，保存 `presetVersionId`、每次运行的
|
||||
`messages` 快照和每个模型的结果。结果不自动评分，人工标记成功 / 失败；标记会落盘。
|
||||
历史运行可在结果卡上方切回，并展开「发送内容」核对实际请求。
|
||||
|
||||
## 6. HTTP 接口
|
||||
|
||||
前缀 `/api/preset-lab/`：
|
||||
|
||||
```text
|
||||
GET presets 预设列表
|
||||
POST presets/create 新建原生预设 + 根版本
|
||||
GET preset?presetId= 读预设文件
|
||||
POST preset/apply 应用到对话页（{force:true} 跳过 error）
|
||||
GET versions?presetId= 版本树
|
||||
POST versions/commit 追加节点（缺省父节点=当前；指定历史节点=分叉）
|
||||
POST versions/amend 覆盖当前节点（保存至当前版本）
|
||||
POST versions/checkout 切换当前节点
|
||||
POST check 结构检查
|
||||
POST preview 按用例组装 messages
|
||||
POST run 多模型运行（可带 preset 草稿；一次跑一个用例）
|
||||
GET cases 使用例列表（全局，不看 presetId）
|
||||
POST cases/save 存单份用例（带 groupId 就写回原文件）
|
||||
POST cases/delete 删单份用例（空文件自动收掉）
|
||||
GET lab-state?presetId= 这个预设上次打开的用例
|
||||
POST lab-state 写 { presetId, selectedCaseId?, previewCaseId? }
|
||||
GET groups 旧接口：测试组列表
|
||||
POST groups/save 旧接口：整组覆盖
|
||||
POST groups/delete 旧接口：删组
|
||||
GET runs?presetId= 运行记录列表
|
||||
GET run?presetId=&runId= 单次运行
|
||||
POST verdict 修改人工标记
|
||||
GET profiles 可选 API 配置
|
||||
```
|
||||
|
||||
## 7. 已知边界
|
||||
|
||||
- 版本树不做自动合并；分叉后各分支独立。
|
||||
- 运行是普通 HTTP 请求，页面会一直等到全部结果返回；没有逐条流式进度。
|
||||
- 「应用到对话页」只改预设文件；要让已有会话立刻使用，去设置页点「启用」。
|
||||
- 测试用例不会自动生成，也不会自动评分。
|
||||
@@ -0,0 +1,116 @@
|
||||
import { collectMacroUsages, findUnclosedMacro } from "../directives/macros.js";
|
||||
import type { PresetPackage } from "../types/preset.js";
|
||||
import { CHAT_HISTORY_MARKER } from "./markers.js";
|
||||
|
||||
/**
|
||||
* 预设编辑器的结构检查（不调用模型）。
|
||||
* error：会导致装配或发送异常；warn：会让预设行为与预期不符，但仍可发送。
|
||||
*/
|
||||
export type AuthoringIssue = {
|
||||
level: "error" | "warn";
|
||||
entryId: string | null;
|
||||
entryName: string;
|
||||
message: string;
|
||||
};
|
||||
|
||||
const VALID_ROLES = new Set(["system", "user", "assistant"]);
|
||||
const FULLWIDTH_OPEN = "｛｛";
|
||||
|
||||
export function checkAuthoringPreset(preset: PresetPackage): AuthoringIssue[] {
|
||||
const issues: AuthoringIssue[] = [];
|
||||
const byId = new Map<string, PresetPackage["prompts"][number]>();
|
||||
const push = (
|
||||
level: AuthoringIssue["level"],
|
||||
entryId: string | null,
|
||||
entryName: string,
|
||||
message: string,
|
||||
) => issues.push({ level, entryId, entryName, message });
|
||||
|
||||
if (preset.prompts.length === 0) {
|
||||
push("error", null, "", "预设没有任何模块");
|
||||
return issues;
|
||||
}
|
||||
|
||||
for (const prompt of preset.prompts) {
|
||||
if (byId.has(prompt.id)) {
|
||||
push("error", prompt.id, prompt.name || prompt.id, "模块 id 重复");
|
||||
continue;
|
||||
}
|
||||
byId.set(prompt.id, prompt);
|
||||
}
|
||||
|
||||
const enabledOrder = preset.promptOrder.filter((o) => o.enabled);
|
||||
if (!enabledOrder.some((o) => o.promptId === CHAT_HISTORY_MARKER)) {
|
||||
push("warn", null, "", "没有启用「对话历史」位置，真实聊天记录不会进入请求");
|
||||
}
|
||||
|
||||
const inOrder = new Set(preset.promptOrder.map((o) => o.promptId));
|
||||
for (const prompt of preset.prompts) {
|
||||
if (!inOrder.has(prompt.id)) {
|
||||
push("warn", prompt.id, prompt.name, "未加入顺序，不会发送");
|
||||
}
|
||||
}
|
||||
|
||||
const seenOrder = new Set<string>();
|
||||
const seenOrderIndex = new Set<number>();
|
||||
const seenMarkers = new Set<string>();
|
||||
for (const item of preset.promptOrder) {
|
||||
if (seenOrder.has(item.promptId)) {
|
||||
push("error", item.promptId, item.promptId, "顺序中重复引用了同一模块");
|
||||
}
|
||||
seenOrder.add(item.promptId);
|
||||
if (seenOrderIndex.has(item.orderIndex)) {
|
||||
push("warn", item.promptId, item.promptId, `顺序号重复：${item.orderIndex}`);
|
||||
}
|
||||
seenOrderIndex.add(item.orderIndex);
|
||||
if (!byId.has(item.promptId)) {
|
||||
push("error", item.promptId, item.promptId, "顺序中引用了不存在的模块");
|
||||
}
|
||||
}
|
||||
|
||||
for (const item of enabledOrder) {
|
||||
const entry = byId.get(item.promptId);
|
||||
if (!entry) continue;
|
||||
const name = entry.name || entry.id;
|
||||
|
||||
if (entry.marker) {
|
||||
if (seenMarkers.has(entry.id)) {
|
||||
push("warn", entry.id, name, "位置在同一顺序中启用了多次，会重复填充");
|
||||
}
|
||||
seenMarkers.add(entry.id);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!VALID_ROLES.has(entry.role)) {
|
||||
push("error", entry.id, name, `角色非法：${String(entry.role)}`);
|
||||
}
|
||||
|
||||
const body = entry.content ?? "";
|
||||
if (!body.trim()) {
|
||||
push("warn", entry.id, name, "模块正文为空，将被跳过");
|
||||
continue;
|
||||
}
|
||||
|
||||
if (body.includes(FULLWIDTH_OPEN)) {
|
||||
push(
|
||||
"warn",
|
||||
entry.id,
|
||||
name,
|
||||
"含全角 ｛｛，运行时不会展开，请改为半角 {{",
|
||||
);
|
||||
}
|
||||
|
||||
const unclosed = findUnclosedMacro(body);
|
||||
if (unclosed) {
|
||||
push("error", entry.id, name, `存在未闭合的 {{ 宏：${unclosed}`);
|
||||
}
|
||||
|
||||
for (const usage of collectMacroUsages(body)) {
|
||||
if (!usage.known) {
|
||||
push("warn", entry.id, name, `未识别宏 ${usage.name}，将原样发出`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return issues;
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { getPresetsDir } from "../config/user-data-dir.js";
|
||||
|
||||
/**
|
||||
* 每个预设在试跑视图里的「上次打开过啥」。
|
||||
*
|
||||
* 用例本身是全局的（`presets/tests/global`），但「这个预设上次用的是哪条用例」
|
||||
* 得按预设记，否则切换预设时选择会跳。存盘而不是塞浏览器缓存：换浏览器、
|
||||
* 重开页面都还是同一条。
|
||||
*
|
||||
* 文件：`<data>/presets/lab-state/<presetId>.json`
|
||||
*/
|
||||
export type PresetLabState = {
|
||||
presetId: string;
|
||||
/** 试跑视图的当前用例（`TestCase.id`） */
|
||||
selectedCaseId: string | null;
|
||||
/** 编辑视图右栏预览用的用例 */
|
||||
previewCaseId: string | null;
|
||||
updatedAt: string;
|
||||
};
|
||||
|
||||
const SAFE_ID = /^[\w\u4e00-\u9fff-]+$/;
|
||||
|
||||
function stateDir(): string {
|
||||
return path.join(getPresetsDir(), "lab-state");
|
||||
}
|
||||
|
||||
function stateFile(presetId: string): string {
|
||||
if (!SAFE_ID.test(presetId)) throw new Error(`非法预设 id：${presetId}`);
|
||||
return path.join(stateDir(), `${presetId}.json`);
|
||||
}
|
||||
|
||||
function emptyState(presetId: string): PresetLabState {
|
||||
return { presetId, selectedCaseId: null, previewCaseId: null, updatedAt: new Date().toISOString() };
|
||||
}
|
||||
|
||||
export function loadPresetLabState(presetId: string): PresetLabState {
|
||||
const file = stateFile(presetId);
|
||||
if (!existsSync(file)) return emptyState(presetId);
|
||||
try {
|
||||
const parsed = JSON.parse(readFileSync(file, "utf8")) as Partial<PresetLabState>;
|
||||
return {
|
||||
presetId,
|
||||
selectedCaseId: typeof parsed.selectedCaseId === "string" ? parsed.selectedCaseId : null,
|
||||
previewCaseId: typeof parsed.previewCaseId === "string" ? parsed.previewCaseId : null,
|
||||
updatedAt: typeof parsed.updatedAt === "string" ? parsed.updatedAt : new Date().toISOString(),
|
||||
};
|
||||
} catch {
|
||||
return emptyState(presetId);
|
||||
}
|
||||
}
|
||||
|
||||
/** 只写传进来的字段，没传的保持原样。 */
|
||||
export function savePresetLabState(
|
||||
presetId: string,
|
||||
patch: { selectedCaseId?: string | null; previewCaseId?: string | null },
|
||||
): PresetLabState {
|
||||
const current = loadPresetLabState(presetId);
|
||||
const next: PresetLabState = {
|
||||
...current,
|
||||
selectedCaseId:
|
||||
patch.selectedCaseId === undefined ? current.selectedCaseId : patch.selectedCaseId,
|
||||
previewCaseId:
|
||||
patch.previewCaseId === undefined ? current.previewCaseId : patch.previewCaseId,
|
||||
updatedAt: new Date().toISOString(),
|
||||
};
|
||||
mkdirSync(stateDir(), { recursive: true });
|
||||
writeFileSync(stateFile(presetId), JSON.stringify(next, null, 2), "utf8");
|
||||
return next;
|
||||
}
|
||||
@@ -0,0 +1,108 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import type { PresetPackage } from "../types/preset.js";
|
||||
|
||||
/**
|
||||
* 工作台新建一份原生预设。
|
||||
* 骨架给一条可编辑规则 + 完整的扩展 marker 顺序，方便直接在页面上
|
||||
* 调整位置；applyExtendedMarkers 调用时仍是幂等的。
|
||||
*/
|
||||
export function createNativePreset(input: {
|
||||
id?: string | null;
|
||||
name?: string | null;
|
||||
} = {}): PresetPackage {
|
||||
const id = input.id?.trim() || `preset-${randomUUID().slice(0, 8)}`;
|
||||
const name = input.name?.trim() || "未命名预设";
|
||||
return {
|
||||
id,
|
||||
name,
|
||||
source: "native",
|
||||
prompts: [
|
||||
{
|
||||
id: "system-rules",
|
||||
name: "写作规则",
|
||||
enabled: true,
|
||||
role: "system",
|
||||
content:
|
||||
"你是中文小说叙事者。按用户输入续写故事，保持人物、设定与前文一致；只输出正文，不解释写作过程。",
|
||||
marker: false,
|
||||
sourceIdentifier: "system-rules",
|
||||
},
|
||||
{
|
||||
id: "charDescription",
|
||||
name: "角色描述",
|
||||
enabled: true,
|
||||
role: "system",
|
||||
content: "",
|
||||
marker: true,
|
||||
sourceIdentifier: "charDescription",
|
||||
},
|
||||
{
|
||||
id: "personaDescription",
|
||||
name: "用户人设",
|
||||
enabled: true,
|
||||
role: "system",
|
||||
content: "",
|
||||
marker: true,
|
||||
sourceIdentifier: "personaDescription",
|
||||
},
|
||||
{
|
||||
id: "worldBookBefore",
|
||||
name: "世界书（历史前）",
|
||||
enabled: true,
|
||||
role: "system",
|
||||
content: "",
|
||||
marker: true,
|
||||
sourceIdentifier: "worldBookBefore",
|
||||
},
|
||||
{
|
||||
id: "chatHistory",
|
||||
name: "对话历史",
|
||||
enabled: true,
|
||||
role: "system",
|
||||
content: "",
|
||||
marker: true,
|
||||
sourceIdentifier: "chatHistory",
|
||||
},
|
||||
{
|
||||
id: "worldBookAfter",
|
||||
name: "世界书（历史后）",
|
||||
enabled: true,
|
||||
role: "system",
|
||||
content: "",
|
||||
marker: true,
|
||||
sourceIdentifier: "worldBookAfter",
|
||||
},
|
||||
{
|
||||
id: "currentTurn",
|
||||
name: "本轮对话",
|
||||
enabled: true,
|
||||
role: "user",
|
||||
content: "",
|
||||
marker: true,
|
||||
sourceIdentifier: "currentTurn",
|
||||
},
|
||||
{
|
||||
id: "postTurn",
|
||||
name: "本轮之后",
|
||||
enabled: true,
|
||||
role: "system",
|
||||
content: "",
|
||||
marker: true,
|
||||
sourceIdentifier: "postTurn",
|
||||
},
|
||||
],
|
||||
promptOrder: [
|
||||
{ promptId: "system-rules", enabled: true, orderIndex: 0 },
|
||||
{ promptId: "charDescription", enabled: true, orderIndex: 1 },
|
||||
{ promptId: "personaDescription", enabled: true, orderIndex: 2 },
|
||||
{ promptId: "worldBookBefore", enabled: true, orderIndex: 3 },
|
||||
{ promptId: "chatHistory", enabled: true, orderIndex: 4 },
|
||||
{ promptId: "worldBookAfter", enabled: true, orderIndex: 5 },
|
||||
{ promptId: "currentTurn", enabled: true, orderIndex: 6 },
|
||||
{ promptId: "postTurn", enabled: true, orderIndex: 7 },
|
||||
],
|
||||
generation: {},
|
||||
unsupported: [],
|
||||
importedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
+7
-2
@@ -19,6 +19,11 @@ function presetPath(id: string): string {
|
||||
return path.join(getPresetsDir(), `${id}.json`);
|
||||
}
|
||||
|
||||
/** Windows 编辑器可能给 JSON 加 BOM；JSON.parse 不认，先剥掉。 */
|
||||
function parsePreset(raw: string): PresetPackage {
|
||||
return JSON.parse(raw.replace(/^\uFEFF/, "")) as PresetPackage;
|
||||
}
|
||||
|
||||
export function listPresets(): PresetPackage[] {
|
||||
ensureUserDataDirs();
|
||||
let files: string[];
|
||||
@@ -33,7 +38,7 @@ export function listPresets(): PresetPackage[] {
|
||||
try {
|
||||
const filePath = path.join(getPresetsDir(), file);
|
||||
const raw = readFileSync(filePath, "utf8");
|
||||
presets.push(withoutReasoningEffort(JSON.parse(raw) as PresetPackage, filePath));
|
||||
presets.push(withoutReasoningEffort(parsePreset(raw), filePath));
|
||||
} catch {
|
||||
/* skip corrupt files */
|
||||
}
|
||||
@@ -45,7 +50,7 @@ export function getPreset(id: string): PresetPackage | null {
|
||||
try {
|
||||
const filePath = presetPath(id);
|
||||
const raw = readFileSync(filePath, "utf8");
|
||||
return withoutReasoningEffort(JSON.parse(raw) as PresetPackage, filePath);
|
||||
return withoutReasoningEffort(parsePreset(raw), filePath);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { existsSync, mkdirSync, readFileSync, readdirSync, unlinkSync, writeFileSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { getPresetsDir } from "../config/user-data-dir.js";
|
||||
|
||||
/**
|
||||
* 测试用例：**全局，不和预设绑定**——一个用例只描述「要测的输入 + 上下文」，
|
||||
* 任何预设都能拿它去跑。存在 `<data>/presets/tests/global/` 下，每组一个 JSON 文件
|
||||
* （沿用 TestCaseGroup 形状，旧的按预设分目录的文件会在第一次访问时并进来）。
|
||||
*
|
||||
* 每个用例只包含变动项：角色名与描述（{{char}}）、世界书、用户输入、可选历史。
|
||||
*/
|
||||
export type TestCaseHistoryItem = { role: "user" | "assistant"; content: string };
|
||||
|
||||
export type TestCase = {
|
||||
id: string;
|
||||
name: string;
|
||||
charName: string;
|
||||
charDescription: string;
|
||||
worldBook: string;
|
||||
userInput: string;
|
||||
history: TestCaseHistoryItem[];
|
||||
};
|
||||
|
||||
export type TestCaseGroup = {
|
||||
id: string;
|
||||
/** 历史字段：现在恒为 GLOBAL_CASES_ID，留着只是为了旧文件能直接读。 */
|
||||
presetId: string;
|
||||
name: string;
|
||||
updatedAt: string;
|
||||
cases: TestCase[];
|
||||
};
|
||||
|
||||
export const GLOBAL_CASES_ID = "global";
|
||||
|
||||
const SAFE_ID = /^[\w\u4e00-\u9fff-]+$/;
|
||||
|
||||
function assertSafe(id: string, label: string): void {
|
||||
if (!SAFE_ID.test(id)) throw new Error(`非法${label}：${id}`);
|
||||
}
|
||||
|
||||
function casesRoot(): string {
|
||||
return path.join(getPresetsDir(), "tests");
|
||||
}
|
||||
|
||||
function globalDir(): string {
|
||||
return path.join(casesRoot(), GLOBAL_CASES_ID);
|
||||
}
|
||||
|
||||
function groupFile(groupId: string): string {
|
||||
assertSafe(groupId, "用例 id");
|
||||
return path.join(globalDir(), `${groupId}.json`);
|
||||
}
|
||||
|
||||
/**
|
||||
* 老版本把用例按预设分目录存（`tests/<presetId>/*.json`）。全局目录第一次被用到时，
|
||||
* 把那些文件并进来（id 冲突就跳过），原文件不删，方便回退。
|
||||
*/
|
||||
function ensureGlobalDir(): string {
|
||||
const dir = globalDir();
|
||||
if (existsSync(dir)) return dir;
|
||||
mkdirSync(dir, { recursive: true });
|
||||
const root = casesRoot();
|
||||
for (const entry of readdirSync(root, { withFileTypes: true })) {
|
||||
if (!entry.isDirectory() || entry.name === GLOBAL_CASES_ID) continue;
|
||||
const legacyDir = path.join(root, entry.name);
|
||||
for (const file of readdirSync(legacyDir).filter((name) => name.endsWith(".json"))) {
|
||||
const target = path.join(dir, file);
|
||||
if (existsSync(target)) continue;
|
||||
try {
|
||||
const group = JSON.parse(readFileSync(path.join(legacyDir, file), "utf8")) as TestCaseGroup;
|
||||
writeFileSync(
|
||||
target,
|
||||
JSON.stringify({ ...group, presetId: GLOBAL_CASES_ID }, null, 2),
|
||||
"utf8",
|
||||
);
|
||||
} catch {
|
||||
/* 坏文件跳过 */
|
||||
}
|
||||
}
|
||||
}
|
||||
return dir;
|
||||
}
|
||||
|
||||
export function listTestGroups(): TestCaseGroup[] {
|
||||
const dir = ensureGlobalDir();
|
||||
const groups: TestCaseGroup[] = [];
|
||||
for (const file of readdirSync(dir).filter((name) => name.endsWith(".json"))) {
|
||||
try {
|
||||
groups.push(JSON.parse(readFileSync(path.join(dir, file), "utf8")) as TestCaseGroup);
|
||||
} catch {
|
||||
/* skip corrupt */
|
||||
}
|
||||
}
|
||||
return groups.sort((a, b) => b.updatedAt.localeCompare(a.updatedAt));
|
||||
}
|
||||
|
||||
export function getTestGroup(groupId: string): TestCaseGroup | null {
|
||||
try {
|
||||
return JSON.parse(readFileSync(groupFile(groupId), "utf8")) as TestCaseGroup;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** 新建或覆盖一组。cases 中缺 id 的会分配 id。 */
|
||||
export function saveTestGroup(input: {
|
||||
id?: string;
|
||||
name: string;
|
||||
cases: Array<Partial<TestCase>>;
|
||||
}): TestCaseGroup {
|
||||
const id = input.id?.trim() || randomUUID().slice(0, 8);
|
||||
const text = (value: unknown): string => (typeof value === "string" ? value : "");
|
||||
const cases: TestCase[] = input.cases.map((c) => ({
|
||||
id: text(c.id).trim() || randomUUID().slice(0, 8),
|
||||
name: text(c.name).trim() || "未命名用例",
|
||||
charName: text(c.charName).trim(),
|
||||
charDescription: text(c.charDescription),
|
||||
worldBook: text(c.worldBook),
|
||||
userInput: text(c.userInput),
|
||||
history: (Array.isArray(c.history) ? c.history : []).filter(
|
||||
(h) => (h.role === "user" || h.role === "assistant") && typeof h.content === "string",
|
||||
),
|
||||
}));
|
||||
const group: TestCaseGroup = {
|
||||
id,
|
||||
presetId: GLOBAL_CASES_ID,
|
||||
name: input.name.trim() || "未命名测试组",
|
||||
updatedAt: new Date().toISOString(),
|
||||
cases,
|
||||
};
|
||||
ensureGlobalDir();
|
||||
writeFileSync(groupFile(id), JSON.stringify(group, null, 2), "utf8");
|
||||
return group;
|
||||
}
|
||||
|
||||
export function deleteTestGroup(groupId: string): void {
|
||||
try {
|
||||
unlinkSync(groupFile(groupId));
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
|
||||
/* ---------- 用例（工作台的展示单位） ----------
|
||||
* 盘上仍是一组一个文件，但界面上不出现「组」：每份用例都是一条独立记录，
|
||||
* 自己存、自己运行。新建用例会为它开一个只装一份用例的组文件。
|
||||
*/
|
||||
|
||||
export type TestCaseRef = { groupId: string; case: TestCase };
|
||||
|
||||
/** 平铺所有用例（按组更新时间排序，组内保持原顺序）。 */
|
||||
export function listTestCases(): TestCaseRef[] {
|
||||
const out: TestCaseRef[] = [];
|
||||
for (const group of listTestGroups()) {
|
||||
for (const item of group.cases) out.push({ groupId: group.id, case: item });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* 保存单份用例：给了 groupId 就写回那个组（组内其他用例不动），
|
||||
* 没给就新建一个只装它的组。
|
||||
*/
|
||||
export function saveTestCase(input: {
|
||||
groupId?: string;
|
||||
case: Partial<TestCase>;
|
||||
}): TestCaseRef {
|
||||
const groupId = input.groupId?.trim();
|
||||
if (!groupId) {
|
||||
const name = typeof input.case.name === "string" ? input.case.name.trim() : "";
|
||||
const group = saveTestGroup({
|
||||
name: name || "未命名用例",
|
||||
cases: [input.case],
|
||||
});
|
||||
return { groupId: group.id, case: group.cases[0]! };
|
||||
}
|
||||
|
||||
const group = getTestGroup(groupId);
|
||||
if (!group) throw new Error(`用例不存在：${groupId}`);
|
||||
const caseId = typeof input.case.id === "string" ? input.case.id.trim() : "";
|
||||
const index = group.cases.findIndex((item) => item.id === caseId);
|
||||
const cases =
|
||||
index >= 0
|
||||
? group.cases.map((item, i) => (i === index ? { ...item, ...input.case } : item))
|
||||
: [...group.cases, input.case];
|
||||
// 只剩一份用例时，组名跟着用例名走，盘上读起来不至于两个名字打架。
|
||||
const single = cases.length === 1;
|
||||
const caseName = typeof input.case.name === "string" ? input.case.name.trim() : "";
|
||||
const saved = saveTestGroup({
|
||||
id: groupId,
|
||||
name: single && caseName ? caseName : group.name,
|
||||
cases,
|
||||
});
|
||||
const wanted = index >= 0 ? caseId : saved.cases[saved.cases.length - 1]!.id;
|
||||
return {
|
||||
groupId,
|
||||
case: saved.cases.find((item) => item.id === wanted) ?? saved.cases[0]!,
|
||||
};
|
||||
}
|
||||
|
||||
/** 删除单份用例；组里没别的人就把组文件一起删掉。 */
|
||||
export function deleteTestCase(groupId: string, caseId: string): void {
|
||||
const group = getTestGroup(groupId);
|
||||
if (!group) return;
|
||||
const cases = group.cases.filter((item) => item.id !== caseId);
|
||||
if (cases.length === 0) deleteTestGroup(groupId);
|
||||
else saveTestGroup({ id: groupId, name: group.name, cases });
|
||||
}
|
||||
@@ -0,0 +1,224 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { getApiProfile, listApiProfiles, profileToLlmConfig } from "../config/api-profiles.js";
|
||||
import { OpenAiCompatibleProvider, type ChatMessage, type TokenUsage } from "../llm/client.js";
|
||||
import { assemblePlayWorkerMessages, shapeFramedMessages } from "./play-frame.js";
|
||||
import { resolveSquashForPreset } from "../llm/preset-wrapper.js";
|
||||
import { checkAuthoringPreset, type AuthoringIssue } from "./authoring-check.js";
|
||||
import type { PresetPackage } from "../types/preset.js";
|
||||
import type { MessageSquashMode } from "../types/preset.js";
|
||||
import type { TestCase } from "./test-cases.js";
|
||||
import type { WorldInfoPack } from "./world-info-pack.js";
|
||||
|
||||
/**
|
||||
* 按真实发送路径组装一个用例的 messages（不调用模型）。
|
||||
* 与线上一致：预设 → assemblePlayWorkerMessages → 消息整形。
|
||||
* 变动的只有 char 描述、世界书、用户输入、历史。
|
||||
*/
|
||||
export function buildCaseMessages(preset: PresetPackage, testCase: TestCase): ChatMessage[] {
|
||||
return shapeFramedMessages(
|
||||
assembleCaseMessages(preset, testCase),
|
||||
resolveSquashForPreset(preset, "follow-preset"),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* 工作台右侧预览：同一组装结果，多告诉页面两件事——
|
||||
* 本次用的整形模式、整形前的条数。左轨是真实顺序，整形后条数变少要能看出来。
|
||||
*/
|
||||
export function buildCasePreview(
|
||||
preset: PresetPackage,
|
||||
testCase: TestCase,
|
||||
): { messages: ChatMessage[]; squash: MessageSquashMode; assembledCount: number } {
|
||||
const framed = assembleCaseMessages(preset, testCase);
|
||||
const squash = resolveSquashForPreset(preset, "follow-preset");
|
||||
return {
|
||||
messages: shapeFramedMessages(framed, squash),
|
||||
squash,
|
||||
assembledCount: framed.length,
|
||||
};
|
||||
}
|
||||
|
||||
/** 装配那一步本身（右栏与试跑共用，整形在调用方决定）。 */
|
||||
function assembleCaseMessages(
|
||||
preset: PresetPackage,
|
||||
testCase: TestCase,
|
||||
): ChatMessage[] {
|
||||
const asText = (value: unknown): string => (typeof value === "string" ? value : "");
|
||||
const historyInput = (Array.isArray(testCase.history) ? testCase.history : []).filter(
|
||||
(item) => item && typeof item === "object",
|
||||
);
|
||||
const history: ChatMessage[] = historyInput.map((h) => ({
|
||||
role: h.role === "assistant" ? "assistant" : "user",
|
||||
content: typeof h.content === "string" ? h.content : "",
|
||||
}));
|
||||
const worldBook = asText(testCase.worldBook);
|
||||
const charDescription = asText(testCase.charDescription);
|
||||
const pack: WorldInfoPack = {
|
||||
worldBookBefore: worldBook.trim()
|
||||
? [{ id: "test-world", name: "世界书", content: worldBook }]
|
||||
: [],
|
||||
history,
|
||||
worldBookAfter: [],
|
||||
turn: asText(testCase.userInput),
|
||||
postTurn: [],
|
||||
card: charDescription.trim() ? { description: charDescription } : undefined,
|
||||
};
|
||||
const framed = assemblePlayWorkerMessages({
|
||||
systemPrompt: "",
|
||||
preset,
|
||||
pack,
|
||||
persona: null,
|
||||
charName: asText(testCase.charName) || null,
|
||||
});
|
||||
return framed;
|
||||
}
|
||||
|
||||
export type RunStatus = "running" | "done" | "error";
|
||||
|
||||
export type CaseRunResult = {
|
||||
profileId: string;
|
||||
profileName: string;
|
||||
model: string;
|
||||
caseId: string;
|
||||
caseName?: string;
|
||||
status: RunStatus;
|
||||
content?: string;
|
||||
error?: string;
|
||||
usage?: TokenUsage;
|
||||
durationMs?: number;
|
||||
/** 手动标记：null 表示未标记 */
|
||||
verdict: "success" | "failure" | null;
|
||||
startedAt: string;
|
||||
finishedAt?: string;
|
||||
};
|
||||
|
||||
export type TestRun = {
|
||||
id: string;
|
||||
presetId: string;
|
||||
presetVersionId: string;
|
||||
groupId: string;
|
||||
createdAt: string;
|
||||
/** 本次运行时的消息快照（每个用例一份），便于事后核对发送内容 */
|
||||
requests: Array<{ caseId: string; messages: ChatMessage[] }>;
|
||||
results: CaseRunResult[];
|
||||
issues: AuthoringIssue[];
|
||||
};
|
||||
|
||||
export const TEST_RUN_TIMEOUT_MS = 180_000;
|
||||
const DEFAULT_CONCURRENCY = 4;
|
||||
|
||||
/** 小并发池：多模型 × 多用例时不要一次全打出去。 */
|
||||
async function runPool(jobs: Array<() => Promise<void>>, limit: number): Promise<void> {
|
||||
const queue = [...jobs];
|
||||
const width = Math.max(1, Math.min(limit, queue.length));
|
||||
const workers = Array.from({ length: width }, async () => {
|
||||
while (queue.length > 0) {
|
||||
const job = queue.shift();
|
||||
if (job) await job();
|
||||
}
|
||||
});
|
||||
await Promise.all(workers);
|
||||
}
|
||||
|
||||
/**
|
||||
* 对一组用例、多个 API 配置并行运行。
|
||||
* 同一用例的同一配置只发一次；各配置之间并行。
|
||||
*/
|
||||
export async function runTestGroup(input: {
|
||||
preset: PresetPackage;
|
||||
presetVersionId: string;
|
||||
presetId: string;
|
||||
groupId: string;
|
||||
cases: TestCase[];
|
||||
profileIds: string[];
|
||||
timeoutMs?: number;
|
||||
concurrency?: number;
|
||||
}): Promise<TestRun> {
|
||||
const profileIds = [...new Set(input.profileIds.map((x) => x.trim()).filter(Boolean))];
|
||||
if (profileIds.length === 0) throw new Error("请至少选择一个 API 配置");
|
||||
if (input.cases.length === 0) throw new Error("测试组中没有用例");
|
||||
|
||||
const profiles = profileIds.map((id) => {
|
||||
const profile = getApiProfile(id);
|
||||
if (!profile?.apiKey?.trim()) throw new Error(`API 配置不可用：${id}`);
|
||||
return profile;
|
||||
});
|
||||
|
||||
const requests = input.cases.map((c) => ({ caseId: c.id, messages: buildCaseMessages(input.preset, c) }));
|
||||
const run: TestRun = {
|
||||
id: randomUUID().slice(0, 8),
|
||||
presetId: input.presetId,
|
||||
presetVersionId: input.presetVersionId,
|
||||
groupId: input.groupId,
|
||||
createdAt: new Date().toISOString(),
|
||||
requests,
|
||||
results: [],
|
||||
issues: checkAuthoringPreset(input.preset),
|
||||
};
|
||||
|
||||
const timeoutMs = Math.max(1_000, input.timeoutMs ?? TEST_RUN_TIMEOUT_MS);
|
||||
const jobs: Array<() => Promise<void>> = [];
|
||||
for (const profile of profiles) {
|
||||
for (const req of requests) {
|
||||
const testCase = input.cases.find((c) => c.id === req.caseId);
|
||||
const slot: CaseRunResult = {
|
||||
profileId: profile.id,
|
||||
profileName: profile.name,
|
||||
model: profile.model,
|
||||
caseId: req.caseId,
|
||||
caseName: testCase?.name,
|
||||
status: "running",
|
||||
verdict: null,
|
||||
startedAt: new Date().toISOString(),
|
||||
};
|
||||
run.results.push(slot);
|
||||
jobs.push(async () => {
|
||||
const started = Date.now();
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
const llm = new OpenAiCompatibleProvider(profileToLlmConfig(profile));
|
||||
const result = await llm.complete(req.messages, {
|
||||
generation: input.preset.generation,
|
||||
caller: `preset-test:${profile.id}`,
|
||||
signal: controller.signal,
|
||||
});
|
||||
slot.content = result.content ?? "";
|
||||
if (result.model) slot.model = result.model;
|
||||
if (result.usage) slot.usage = result.usage;
|
||||
slot.status = "done";
|
||||
} catch (err) {
|
||||
slot.status = "error";
|
||||
if (controller.signal.aborted) {
|
||||
slot.error = `超时：${Math.round(timeoutMs / 1000)} 秒内没有完成`;
|
||||
} else {
|
||||
slot.error = err instanceof Error ? err.message : String(err);
|
||||
}
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
slot.durationMs = Date.now() - started;
|
||||
slot.finishedAt = new Date().toISOString();
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
await runPool(jobs, input.concurrency ?? DEFAULT_CONCURRENCY);
|
||||
return run;
|
||||
}
|
||||
|
||||
export function listRunnableProfiles(): Array<{ id: string; name: string; model: string; hasKey: boolean }> {
|
||||
return listApiProfiles().map((p) => ({
|
||||
id: p.id,
|
||||
name: p.name,
|
||||
model: p.model,
|
||||
hasKey: Boolean(p.apiKey?.trim()),
|
||||
}));
|
||||
}
|
||||
|
||||
/** 手动标记结果：只保存标签，不做自动评分。 */
|
||||
export function setVerdict(run: TestRun, profileId: string, caseId: string, verdict: "success" | "failure" | null): TestRun {
|
||||
const slot = run.results.find((r) => r.profileId === profileId && r.caseId === caseId);
|
||||
if (!slot) throw new Error("结果不存在");
|
||||
slot.verdict = verdict;
|
||||
return run;
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { getPresetsDir } from "../config/user-data-dir.js";
|
||||
import type { TestRun } from "./test-run.js";
|
||||
|
||||
/** 运行记录：presets/runs/<presetId>/<runId>.json，保存预设版本 id 以便追溯。 */
|
||||
const SAFE_ID = /^[\w\u4e00-\u9fff-]+$/;
|
||||
|
||||
function runDir(presetId: string): string {
|
||||
if (!SAFE_ID.test(presetId)) throw new Error(`非法预设 id：${presetId}`);
|
||||
return path.join(getPresetsDir(), "runs", presetId);
|
||||
}
|
||||
|
||||
function runFile(presetId: string, runId: string): string {
|
||||
if (!SAFE_ID.test(runId)) throw new Error(`非法运行 id：${runId}`);
|
||||
return path.join(runDir(presetId), `${runId}.json`);
|
||||
}
|
||||
|
||||
export function saveTestRun(run: TestRun): void {
|
||||
mkdirSync(runDir(run.presetId), { recursive: true });
|
||||
writeFileSync(runFile(run.presetId, run.id), JSON.stringify(run, null, 2), "utf8");
|
||||
}
|
||||
|
||||
export function getTestRun(presetId: string, runId: string): TestRun | null {
|
||||
try {
|
||||
return JSON.parse(readFileSync(runFile(presetId, runId), "utf8")) as TestRun;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function listTestRuns(presetId: string): TestRun[] {
|
||||
const dir = runDir(presetId);
|
||||
if (!existsSync(dir)) return [];
|
||||
const out: TestRun[] = [];
|
||||
for (const file of readdirSync(dir).filter((f) => f.endsWith(".json"))) {
|
||||
try {
|
||||
out.push(JSON.parse(readFileSync(path.join(dir, file), "utf8")) as TestRun);
|
||||
} catch {
|
||||
/* skip */
|
||||
}
|
||||
}
|
||||
return out.sort((a, b) => b.createdAt.localeCompare(a.createdAt));
|
||||
}
|
||||
|
||||
/** 修改某个结果的手动标记并落盘。 */
|
||||
export function updateVerdict(
|
||||
presetId: string,
|
||||
runId: string,
|
||||
profileId: string,
|
||||
caseId: string,
|
||||
verdict: "success" | "failure" | null,
|
||||
): TestRun {
|
||||
const run = getTestRun(presetId, runId);
|
||||
if (!run) throw new Error("运行记录不存在");
|
||||
const slot = run.results.find((r) => r.profileId === profileId && r.caseId === caseId);
|
||||
if (!slot) throw new Error("结果不存在");
|
||||
slot.verdict = verdict;
|
||||
saveTestRun(run);
|
||||
return run;
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { getPresetsDir } from "../config/user-data-dir.js";
|
||||
import type { PresetPackage } from "../types/preset.js";
|
||||
|
||||
/**
|
||||
* 预设版本树：每个预设一个 JSON 文件，节点是完整快照。
|
||||
* - 保存：在指定父节点（缺省为当前节点）下追加子节点；
|
||||
* - 分叉：父节点选历史中的任意一个，形成新分支，原分支不动；
|
||||
* - 追溯：沿 parentId 往上走即为修改历史。
|
||||
* 不依赖 git，不做远程同步。
|
||||
*/
|
||||
export type PresetVersionNode = {
|
||||
id: string;
|
||||
parentId: string | null;
|
||||
createdAt: string;
|
||||
/** 被「保存至当前版本」覆盖过的时间；没改过就不写 */
|
||||
updatedAt?: string;
|
||||
note: string;
|
||||
preset: PresetPackage;
|
||||
};
|
||||
|
||||
export type PresetVersionTree = {
|
||||
presetId: string;
|
||||
currentId: string | null;
|
||||
nodes: PresetVersionNode[];
|
||||
};
|
||||
|
||||
const SAFE_ID = /^[\w\u4e00-\u9fff-]+$/;
|
||||
|
||||
function versionsDir(): string {
|
||||
return path.join(getPresetsDir(), "versions");
|
||||
}
|
||||
|
||||
function treeFile(presetId: string): string {
|
||||
if (!SAFE_ID.test(presetId)) throw new Error(`非法预设 id：${presetId}`);
|
||||
return path.join(versionsDir(), `${presetId}.json`);
|
||||
}
|
||||
|
||||
function writeTree(tree: PresetVersionTree): void {
|
||||
mkdirSync(versionsDir(), { recursive: true });
|
||||
writeFileSync(treeFile(tree.presetId), JSON.stringify(tree, null, 2), "utf8");
|
||||
}
|
||||
|
||||
export function loadVersionTree(presetId: string): PresetVersionTree {
|
||||
const file = treeFile(presetId);
|
||||
if (!existsSync(file)) return { presetId, currentId: null, nodes: [] };
|
||||
return JSON.parse(readFileSync(file, "utf8")) as PresetVersionTree;
|
||||
}
|
||||
|
||||
export function getVersionNode(
|
||||
presetId: string,
|
||||
nodeId: string,
|
||||
): PresetVersionNode | null {
|
||||
return loadVersionTree(presetId).nodes.find((n) => n.id === nodeId) ?? null;
|
||||
}
|
||||
|
||||
/** 从根到指定节点的路径（含两端），即该版本的修改历史。 */
|
||||
export function versionLineage(
|
||||
tree: PresetVersionTree,
|
||||
nodeId: string,
|
||||
): PresetVersionNode[] {
|
||||
const byId = new Map(tree.nodes.map((n) => [n.id, n]));
|
||||
const out: PresetVersionNode[] = [];
|
||||
let cursor = byId.get(nodeId) ?? null;
|
||||
const guard = new Set<string>();
|
||||
while (cursor && !guard.has(cursor.id)) {
|
||||
guard.add(cursor.id);
|
||||
out.unshift(cursor);
|
||||
cursor = cursor.parentId ? byId.get(cursor.parentId) ?? null : null;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* 提交一个版本。
|
||||
* parentId 缺省为当前节点；显式传入历史节点即为分叉。
|
||||
* 树为空时 parentId 必须为 null（建根）；已有根时不允许再建第二个根。
|
||||
*/
|
||||
export function commitVersion(input: {
|
||||
presetId: string;
|
||||
preset: PresetPackage;
|
||||
note?: string;
|
||||
parentId?: string | null;
|
||||
}): PresetVersionTree {
|
||||
const tree = loadVersionTree(input.presetId);
|
||||
const parentId =
|
||||
input.parentId !== undefined ? input.parentId : tree.currentId;
|
||||
|
||||
if (parentId === null) {
|
||||
if (tree.nodes.length > 0) throw new Error("已有版本，不能再新建根版本");
|
||||
} else if (!tree.nodes.some((n) => n.id === parentId)) {
|
||||
throw new Error(`父版本不存在：${parentId}`);
|
||||
}
|
||||
|
||||
let id = randomUUID().slice(0, 8);
|
||||
while (tree.nodes.some((n) => n.id === id)) id = randomUUID().slice(0, 8);
|
||||
const node: PresetVersionNode = {
|
||||
id,
|
||||
parentId,
|
||||
createdAt: new Date().toISOString(),
|
||||
note: (input.note ?? "").trim(),
|
||||
preset: structuredClone(input.preset),
|
||||
};
|
||||
tree.nodes.push(node);
|
||||
tree.currentId = node.id;
|
||||
writeTree(tree);
|
||||
return tree;
|
||||
}
|
||||
|
||||
/**
|
||||
* 保存到当前版本（amend）：把快照直接换掉 `currentId` 那个节点，不产生新节点。
|
||||
* 还没有任何版本时退化成建根。
|
||||
*/
|
||||
export function amendCurrentVersion(input: {
|
||||
presetId: string;
|
||||
preset: PresetPackage;
|
||||
note?: string;
|
||||
}): PresetVersionTree {
|
||||
const tree = loadVersionTree(input.presetId);
|
||||
if (!tree.currentId) return commitVersion({ ...input, parentId: null });
|
||||
const node = tree.nodes.find((item) => item.id === tree.currentId);
|
||||
if (!node) throw new Error(`当前版本不存在：${tree.currentId}`);
|
||||
node.preset = structuredClone(input.preset);
|
||||
const note = (input.note ?? "").trim();
|
||||
if (note) node.note = note;
|
||||
node.updatedAt = new Date().toISOString();
|
||||
writeTree(tree);
|
||||
return tree;
|
||||
}
|
||||
|
||||
/** 切换当前编辑位置（不产生新节点）。 */
|
||||
export function checkoutVersion(presetId: string, nodeId: string): PresetVersionTree {
|
||||
const tree = loadVersionTree(presetId);
|
||||
if (!tree.nodes.some((n) => n.id === nodeId)) {
|
||||
throw new Error(`版本不存在：${nodeId}`);
|
||||
}
|
||||
tree.currentId = nodeId;
|
||||
writeTree(tree);
|
||||
return tree;
|
||||
}
|
||||
|
||||
export function deleteVersionTree(presetId: string): void {
|
||||
try {
|
||||
rmSync(treeFile(presetId), { force: true });
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,405 @@
|
||||
import type { IncomingMessage, ServerResponse } from "node:http";
|
||||
import { getPreset, listPresets, savePreset } from "../preset/store.js";
|
||||
import { checkAuthoringPreset } from "../preset/authoring-check.js";
|
||||
import { createNativePreset } from "../preset/new-preset.js";
|
||||
import { amendCurrentVersion, checkoutVersion, commitVersion, getVersionNode, loadVersionTree } from "../preset/versions.js";
|
||||
import { deleteTestGroup, listTestCases, listTestGroups, saveTestCase, saveTestGroup, deleteTestCase } from "../preset/test-cases.js";
|
||||
import { loadPresetLabState, savePresetLabState } from "../preset/lab-state.js";
|
||||
import { buildCasePreview, listRunnableProfiles, runTestGroup } from "../preset/test-run.js";
|
||||
import { getTestRun, listTestRuns, saveTestRun, updateVerdict } from "../preset/test-runs.js";
|
||||
import { setActivePresetId } from "../config/settings.js";
|
||||
import type { PresetPackage } from "../types/preset.js";
|
||||
import type { TestCase } from "../preset/test-cases.js";
|
||||
|
||||
const SAFE_ID = /^[\w\u4e00-\u9fff-]+$/;
|
||||
|
||||
function assertSafeId(id: string, label: string): string {
|
||||
const trimmed = id.trim();
|
||||
if (!SAFE_ID.test(trimmed)) throw new Error(`非法${label}：${id}`);
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
function presetOf(body: Record<string, unknown>): PresetPackage {
|
||||
const preset = body.preset;
|
||||
if (!preset || typeof preset !== "object") throw new Error("缺少 preset");
|
||||
const typed = preset as PresetPackage;
|
||||
assertSafeId(typed.id, "预设 id");
|
||||
return typed;
|
||||
}
|
||||
|
||||
async function readBody(req: IncomingMessage): Promise<string> {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of req) {
|
||||
chunks.push(chunk as Buffer);
|
||||
}
|
||||
return Buffer.concat(chunks).toString("utf8");
|
||||
}
|
||||
|
||||
function json(res: ServerResponse, status: number, data: unknown): void {
|
||||
res.writeHead(status, {
|
||||
"Content-Type": "application/json; charset=utf-8",
|
||||
"Cache-Control": "no-store",
|
||||
});
|
||||
res.end(JSON.stringify(data));
|
||||
}
|
||||
|
||||
async function readJson(req: IncomingMessage): Promise<Record<string, unknown>> {
|
||||
const raw = await readBody(req);
|
||||
if (!raw.trim()) return {};
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
|
||||
throw new Error("请求体必须是 JSON 对象");
|
||||
}
|
||||
return parsed as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function str(body: Record<string, unknown>, key: string): string {
|
||||
const value = body[key];
|
||||
return typeof value === "string" ? value : "";
|
||||
}
|
||||
|
||||
function strList(body: Record<string, unknown>, key: string): string[] {
|
||||
const value = body[key];
|
||||
if (!Array.isArray(value)) return [];
|
||||
return value.filter((x): x is string => typeof x === "string");
|
||||
}
|
||||
|
||||
function numberOrUndefined(value: unknown): number | undefined {
|
||||
const n = typeof value === "number" ? value : Number(value);
|
||||
return Number.isFinite(n) ? n : undefined;
|
||||
}
|
||||
|
||||
/** 取运行时用的预设：优先版本节点快照，其次预设文件。 */
|
||||
function resolveLabPreset(presetId: string, versionId: string): PresetPackage {
|
||||
const id = assertSafeId(presetId, "预设 id");
|
||||
if (versionId) {
|
||||
const node = getVersionNode(id, versionId);
|
||||
if (node) return node.preset;
|
||||
throw new Error(`版本不存在：${versionId}`);
|
||||
}
|
||||
const preset = getPreset(id);
|
||||
if (!preset) throw new Error(`预设不存在：${id}`);
|
||||
return preset;
|
||||
}
|
||||
|
||||
/**
|
||||
* 预设工作台 API。前缀 /api/preset-lab/。
|
||||
* 只做编辑、检查、预览、测试；不做自动评分。
|
||||
*/
|
||||
export async function handlePresetLabApi(
|
||||
req: IncomingMessage,
|
||||
res: ServerResponse,
|
||||
pathname: string,
|
||||
searchParams: URLSearchParams,
|
||||
): Promise<boolean> {
|
||||
if (!pathname.startsWith("/api/preset-lab/")) return false;
|
||||
const route = pathname.slice("/api/preset-lab/".length);
|
||||
const method = req.method ?? "GET";
|
||||
|
||||
try {
|
||||
if (method === "GET" && route === "presets") {
|
||||
json(res, 200, {
|
||||
presets: listPresets().map((p) => ({
|
||||
id: p.id,
|
||||
name: p.name,
|
||||
source: p.source,
|
||||
promptCount: p.prompts.length,
|
||||
})),
|
||||
});
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "presets/create") {
|
||||
const body = await readJson(req);
|
||||
const requestedId = str(body, "id").trim();
|
||||
if (requestedId) assertSafeId(requestedId, "预设 id");
|
||||
const preset = createNativePreset({
|
||||
id: requestedId || null,
|
||||
name: str(body, "name") || null,
|
||||
});
|
||||
if (getPreset(preset.id)) throw new Error(`预设已存在：${preset.id}`);
|
||||
if (loadVersionTree(preset.id).nodes.length > 0) {
|
||||
throw new Error(`版本树已存在：${preset.id}`);
|
||||
}
|
||||
savePreset(preset);
|
||||
const tree = commitVersion({
|
||||
presetId: preset.id,
|
||||
preset,
|
||||
note: str(body, "note") || "新建预设",
|
||||
});
|
||||
json(res, 201, { preset, tree });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "GET" && route === "profiles") {
|
||||
json(res, 200, { profiles: listRunnableProfiles() });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "GET" && route === "preset") {
|
||||
const presetId = assertSafeId(searchParams.get("presetId") ?? "", "预设 id");
|
||||
const preset = getPreset(presetId);
|
||||
if (!preset) {
|
||||
json(res, 404, { error: `预设不存在：${presetId}` });
|
||||
return true;
|
||||
}
|
||||
json(res, 200, { preset });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "preset") {
|
||||
const body = await readJson(req);
|
||||
const preset = presetOf(body);
|
||||
const saved = commitVersion({
|
||||
presetId: preset.id,
|
||||
preset,
|
||||
note: str(body, "note"),
|
||||
parentId: body.parentId === null ? null : typeof body.parentId === "string" ? body.parentId : undefined,
|
||||
});
|
||||
json(res, 200, saved);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "preset/apply") {
|
||||
const body = await readJson(req);
|
||||
const preset = presetOf(body);
|
||||
const issues = checkAuthoringPreset(preset);
|
||||
const errors = issues.filter((issue) => issue.level === "error");
|
||||
if (errors.length > 0 && body.force !== true) {
|
||||
const head = errors
|
||||
.slice(0, 3)
|
||||
.map((issue) => `${issue.entryName ? issue.entryName + "：" : ""}${issue.message}`)
|
||||
.join("；");
|
||||
throw new Error(`结构检查有 ${errors.length} 个错误，先修正或强制写回：${head}`);
|
||||
}
|
||||
const saved = savePreset(structuredClone(preset));
|
||||
const activate = body.activate === true;
|
||||
if (activate) setActivePresetId(saved.id);
|
||||
json(res, 200, {
|
||||
ok: true,
|
||||
preset: saved,
|
||||
issues,
|
||||
activePresetId: activate ? saved.id : null,
|
||||
});
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "GET" && route === "versions") {
|
||||
const presetId = assertSafeId(searchParams.get("presetId") ?? "", "预设 id");
|
||||
json(res, 200, loadVersionTree(presetId));
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "versions/commit") {
|
||||
const body = await readJson(req);
|
||||
const preset = presetOf(body);
|
||||
const bodyPresetId = str(body, "presetId");
|
||||
if (bodyPresetId && bodyPresetId !== preset.id) {
|
||||
throw new Error(`presetId 与 preset.id 不一致：${bodyPresetId} / ${preset.id}`);
|
||||
}
|
||||
const tree = commitVersion({
|
||||
presetId: preset.id,
|
||||
preset,
|
||||
note: str(body, "note"),
|
||||
parentId: body.parentId === null ? null : typeof body.parentId === "string" ? body.parentId : undefined,
|
||||
});
|
||||
json(res, 200, tree);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "versions/amend") {
|
||||
const body = await readJson(req);
|
||||
const preset = presetOf(body);
|
||||
const tree = amendCurrentVersion({
|
||||
presetId: preset.id,
|
||||
preset,
|
||||
note: str(body, "note"),
|
||||
});
|
||||
json(res, 200, tree);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "versions/checkout") {
|
||||
const body = await readJson(req);
|
||||
const tree = checkoutVersion(
|
||||
assertSafeId(str(body, "presetId"), "预设 id"),
|
||||
assertSafeId(str(body, "nodeId"), "版本 id"),
|
||||
);
|
||||
json(res, 200, tree);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "check") {
|
||||
const body = await readJson(req);
|
||||
json(res, 200, { issues: checkAuthoringPreset(presetOf(body)) });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "GET" && route === "groups") {
|
||||
json(res, 200, { groups: listTestGroups() });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "groups/save") {
|
||||
const body = await readJson(req);
|
||||
const cases = Array.isArray(body.cases) ? (body.cases as Array<Partial<TestCase>>) : [];
|
||||
const group = saveTestGroup({
|
||||
id: str(body, "id") || undefined,
|
||||
name: str(body, "name"),
|
||||
cases,
|
||||
});
|
||||
json(res, 200, group);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "groups/delete") {
|
||||
const body = await readJson(req);
|
||||
deleteTestGroup(assertSafeId(str(body, "id"), "测试组 id"));
|
||||
json(res, 200, { ok: true });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "GET" && route === "cases") {
|
||||
// 用例是全局的，不看 presetId；带上也无害
|
||||
json(res, 200, { cases: listTestCases() });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "GET" && route === "lab-state") {
|
||||
const presetId = assertSafeId(searchParams.get("presetId") ?? "", "预设 id");
|
||||
json(res, 200, loadPresetLabState(presetId));
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "lab-state") {
|
||||
const body = await readJson(req);
|
||||
const presetId = assertSafeId(str(body, "presetId"), "预设 id");
|
||||
const asId = (value: unknown): string | null | undefined => {
|
||||
if (value === undefined) return undefined;
|
||||
if (value === null) return null;
|
||||
return typeof value === "string" && value.trim() ? value.trim() : null;
|
||||
};
|
||||
json(
|
||||
res,
|
||||
200,
|
||||
savePresetLabState(presetId, {
|
||||
selectedCaseId: asId(body.selectedCaseId),
|
||||
previewCaseId: asId(body.previewCaseId),
|
||||
}),
|
||||
);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "cases/save") {
|
||||
const body = await readJson(req);
|
||||
const testCase = body.case;
|
||||
if (!testCase || typeof testCase !== "object") throw new Error("缺少 case");
|
||||
const saved = saveTestCase({
|
||||
groupId: str(body, "groupId") || undefined,
|
||||
case: testCase as Partial<TestCase>,
|
||||
});
|
||||
json(res, 200, saved);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "cases/delete") {
|
||||
const body = await readJson(req);
|
||||
deleteTestCase(
|
||||
assertSafeId(str(body, "groupId"), "用例 id"),
|
||||
str(body, "caseId"),
|
||||
);
|
||||
json(res, 200, { ok: true });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "preview") {
|
||||
const body = await readJson(req);
|
||||
const preset = presetOf(body);
|
||||
const testCase = body.testCase as TestCase | undefined;
|
||||
if (!testCase || typeof testCase !== "object") throw new Error("缺少 testCase");
|
||||
json(res, 200, buildCasePreview(preset, testCase));
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "run") {
|
||||
const body = await readJson(req);
|
||||
const presetId = assertSafeId(str(body, "presetId"), "预设 id");
|
||||
const versionId = str(body, "presetVersionId");
|
||||
const groupId = assertSafeId(str(body, "groupId"), "测试组 id");
|
||||
const caseIds = strList(body, "caseIds");
|
||||
const profileIds = strList(body, "profileIds");
|
||||
// 工作台允许直接跑当前草稿；不传 preset 时才退回已保存的版本/文件。
|
||||
const draft = body.preset;
|
||||
const preset =
|
||||
draft && typeof draft === "object"
|
||||
? (() => {
|
||||
const typed = draft as PresetPackage;
|
||||
assertSafeId(typed.id, "预设 id");
|
||||
if (typed.id !== presetId) throw new Error("preset.id 与 presetId 不一致");
|
||||
return typed;
|
||||
})()
|
||||
: resolveLabPreset(presetId, versionId);
|
||||
|
||||
const groups = listTestGroups();
|
||||
const group = groups.find((g) => g.id === groupId);
|
||||
if (!group) throw new Error(`测试组不存在：${groupId}`);
|
||||
const cases = caseIds.length > 0 ? group.cases.filter((c) => caseIds.includes(c.id)) : group.cases;
|
||||
if (cases.length === 0) throw new Error("没有选中任何用例");
|
||||
|
||||
const run = await runTestGroup({
|
||||
preset,
|
||||
presetId,
|
||||
presetVersionId: versionId,
|
||||
groupId,
|
||||
cases,
|
||||
profileIds,
|
||||
timeoutMs: numberOrUndefined(body.timeoutMs),
|
||||
concurrency: numberOrUndefined(body.concurrency),
|
||||
});
|
||||
saveTestRun(run);
|
||||
json(res, 200, run);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "GET" && route === "runs") {
|
||||
json(res, 200, {
|
||||
runs: listTestRuns(assertSafeId(searchParams.get("presetId") ?? "", "预设 id")),
|
||||
});
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "GET" && route === "run") {
|
||||
const run = getTestRun(
|
||||
assertSafeId(searchParams.get("presetId") ?? "", "预设 id"),
|
||||
assertSafeId(searchParams.get("runId") ?? "", "运行 id"),
|
||||
);
|
||||
if (!run) {
|
||||
json(res, 404, { error: "运行记录不存在" });
|
||||
return true;
|
||||
}
|
||||
json(res, 200, run);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (method === "POST" && route === "verdict") {
|
||||
const body = await readJson(req);
|
||||
const raw = body.verdict;
|
||||
const verdict = raw === "success" || raw === "failure" ? raw : null;
|
||||
const run = updateVerdict(
|
||||
assertSafeId(str(body, "presetId"), "预设 id"),
|
||||
assertSafeId(str(body, "runId"), "运行 id"),
|
||||
str(body, "profileId"),
|
||||
str(body, "caseId"),
|
||||
verdict,
|
||||
);
|
||||
json(res, 200, run);
|
||||
return true;
|
||||
}
|
||||
|
||||
json(res, 404, { error: `未知接口：${pathname}` });
|
||||
return true;
|
||||
} catch (err) {
|
||||
json(res, 400, { error: err instanceof Error ? err.message : String(err) });
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -8,6 +8,7 @@ import { handleSettingsApi } from "./settings-handlers.js";
|
||||
import { handleBooksApi } from "./book-handlers.js";
|
||||
import { handleStatsApi } from "./stats-handlers.js";
|
||||
import { handleDirectChatApi } from "./direct-chat-handlers.js";
|
||||
import { handlePresetLabApi } from "./preset-lab-handlers.js";
|
||||
import { syncOpenRouterPrices, type PriceWatchItem } from "../stats/pricing.js";
|
||||
import { ensureConsoleUtf8, getRuntimeLogPath } from "../log.js";
|
||||
|
||||
@@ -42,10 +43,11 @@ async function serveStatic(res: ServerResponse, filePath: string): Promise<void>
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": types[ext] ?? "application/octet-stream",
|
||||
};
|
||||
// 开发期：避免 ES module（agent-ui.js 等）被强缓存，改完 import 仍吃旧脚本导致主区空白
|
||||
if (ext === ".html" || ext === ".js" || ext === ".css") {
|
||||
headers["Cache-Control"] = "no-cache, must-revalidate";
|
||||
}
|
||||
// 开发期：页面与脚本一律不缓存。`no-cache` 还允许浏览器拿着旧副本去问一下，
|
||||
// 换版本后容易继续吃旧 JS；这里直接 `no-store`，刷新即最新。
|
||||
headers["Cache-Control"] = "no-store, must-revalidate";
|
||||
headers["Pragma"] = "no-cache";
|
||||
headers["Expires"] = "0";
|
||||
res.writeHead(200, headers);
|
||||
res.end(content);
|
||||
}
|
||||
@@ -81,6 +83,25 @@ const server = createServer(async (req, res) => {
|
||||
"GET /api/modules",
|
||||
"GET /api/direct-chat/shells",
|
||||
"POST /api/direct-chat/ephemeral/complete",
|
||||
"GET /api/preset-lab/presets",
|
||||
"POST /api/preset-lab/presets/create",
|
||||
"GET /api/preset-lab/preset",
|
||||
"POST /api/preset-lab/preset/apply",
|
||||
"GET /api/preset-lab/versions",
|
||||
"POST /api/preset-lab/versions/commit",
|
||||
"POST /api/preset-lab/versions/checkout",
|
||||
"POST /api/preset-lab/check",
|
||||
"POST /api/preset-lab/preview",
|
||||
"GET /api/preset-lab/groups",
|
||||
"POST /api/preset-lab/groups/save",
|
||||
"POST /api/preset-lab/groups/delete",
|
||||
"GET /api/preset-lab/cases",
|
||||
"POST /api/preset-lab/cases/save",
|
||||
"POST /api/preset-lab/cases/delete",
|
||||
"POST /api/preset-lab/run",
|
||||
"GET /api/preset-lab/runs",
|
||||
"GET /api/preset-lab/run",
|
||||
"POST /api/preset-lab/verdict",
|
||||
"GET /api/stats/tokens",
|
||||
"GET /api/stats/pricing",
|
||||
"GET /api/stats/pricing/openrouter",
|
||||
@@ -104,6 +125,10 @@ const server = createServer(async (req, res) => {
|
||||
return;
|
||||
}
|
||||
|
||||
if (await handlePresetLabApi(req, res, url.pathname, url.searchParams)) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (await handleStatsApi(req, res, url.pathname, url.searchParams)) {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import type { PresetPackage } from "../src/types/preset.js";
|
||||
import { checkAuthoringPreset } from "../src/preset/authoring-check.js";
|
||||
import {
|
||||
checkoutVersion,
|
||||
commitVersion,
|
||||
loadVersionTree,
|
||||
versionLineage,
|
||||
} from "../src/preset/versions.js";
|
||||
|
||||
function preset(id: string, content = "正文"): PresetPackage {
|
||||
return {
|
||||
id,
|
||||
name: id,
|
||||
source: "native",
|
||||
prompts: [
|
||||
{ id: "sys", name: "规则", enabled: true, role: "system", content, marker: false, sourceIdentifier: "sys" },
|
||||
{ id: "chatHistory", name: "对话历史", enabled: true, role: "system", content: "", marker: true, sourceIdentifier: "chatHistory" },
|
||||
],
|
||||
promptOrder: [
|
||||
{ promptId: "sys", enabled: true, orderIndex: 0 },
|
||||
{ promptId: "chatHistory", enabled: true, orderIndex: 1 },
|
||||
],
|
||||
generation: {},
|
||||
unsupported: [],
|
||||
importedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
let dir = "";
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(path.join(os.tmpdir(), "preset-versions-"));
|
||||
process.env.WRITING_AGENT_DATA_DIR = dir;
|
||||
});
|
||||
afterEach(() => {
|
||||
delete process.env.WRITING_AGENT_DATA_DIR;
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe("版本树", () => {
|
||||
it("线性保存并可追溯", () => {
|
||||
commitVersion({ presetId: "p1", preset: preset("p1", "v1") });
|
||||
const tree = commitVersion({ presetId: "p1", preset: preset("p1", "v2") });
|
||||
const lineage = versionLineage(tree, tree.currentId!);
|
||||
expect(lineage.map((n) => n.preset.prompts[0]!.content)).toEqual(["v1", "v2"]);
|
||||
});
|
||||
|
||||
it("从历史节点分叉，原分支保持不变", () => {
|
||||
const first = commitVersion({ presetId: "p2", preset: preset("p2", "a") });
|
||||
const rootId = first.currentId!;
|
||||
commitVersion({ presetId: "p2", preset: preset("p2", "b") });
|
||||
const forked = commitVersion({ presetId: "p2", preset: preset("p2", "c"), parentId: rootId });
|
||||
expect(forked.nodes).toHaveLength(3);
|
||||
const branch = forked.nodes.filter((n) => n.parentId === rootId);
|
||||
expect(branch).toHaveLength(2);
|
||||
expect(versionLineage(forked, forked.currentId!)[0]!.id).toBe(rootId);
|
||||
});
|
||||
|
||||
it("不允许再建第二个根，也不允许挂到不存在的父节点", () => {
|
||||
commitVersion({ presetId: "p3", preset: preset("p3") });
|
||||
expect(() => commitVersion({ presetId: "p3", preset: preset("p3"), parentId: null })).toThrow();
|
||||
expect(() => commitVersion({ presetId: "p3", preset: preset("p3"), parentId: "missing" })).toThrow();
|
||||
});
|
||||
|
||||
it("checkout 只移动当前位置，不新增节点", () => {
|
||||
const tree = commitVersion({ presetId: "p4", preset: preset("p4") });
|
||||
const rootId = tree.currentId!;
|
||||
commitVersion({ presetId: "p4", preset: preset("p4", "x") });
|
||||
const moved = checkoutVersion("p4", rootId);
|
||||
expect(moved.currentId).toBe(rootId);
|
||||
expect(loadVersionTree("p4").nodes).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("拒绝非法预设 id（防路径穿越）", () => {
|
||||
expect(() => loadVersionTree("../evil")).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe("结构检查", () => {
|
||||
it("正常预设没有 error", () => {
|
||||
const issues = checkAuthoringPreset(preset("ok"));
|
||||
expect(issues.filter((i) => i.level === "error")).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("全角 ｛｛ 给 warn 而不是 error", () => {
|
||||
const issues = checkAuthoringPreset(preset("fw", "你好 ｛｛user｛｛"));
|
||||
expect(issues.some((i) => i.level === "warn" && i.message.includes("全角"))).toBe(true);
|
||||
expect(issues.some((i) => i.level === "error")).toBe(false);
|
||||
});
|
||||
|
||||
it("未闭合的 {{ 为 error", () => {
|
||||
const issues = checkAuthoringPreset(preset("open", "{{setvar::a::1"));
|
||||
expect(issues.some((i) => i.level === "error" && i.message.includes("未闭合"))).toBe(true);
|
||||
});
|
||||
|
||||
it("未启用对话历史位置给 warn", () => {
|
||||
const p = preset("nohist");
|
||||
p.promptOrder = p.promptOrder.filter((o) => o.promptId !== "chatHistory");
|
||||
const issues = checkAuthoringPreset(p);
|
||||
expect(issues.some((i) => i.message.includes("对话历史"))).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,146 @@
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { Readable } from "node:stream";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import type { IncomingMessage, ServerResponse } from "node:http";
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import { savePreset } from "../src/preset/store.js";
|
||||
import { handlePresetLabApi } from "../src/server/preset-lab-handlers.js";
|
||||
import type { PresetPackage } from "../src/types/preset.js";
|
||||
|
||||
function preset(id = "lab"): PresetPackage {
|
||||
return {
|
||||
id,
|
||||
name: "工作台预设",
|
||||
source: "native",
|
||||
prompts: [
|
||||
{ id: "sys", name: "规则", enabled: true, role: "system", content: "\u4f60\u662f{{char}}", marker: false, sourceIdentifier: "sys" },
|
||||
{ id: "chatHistory", name: "历史", enabled: true, role: "system", content: "", marker: true, sourceIdentifier: "chatHistory" },
|
||||
],
|
||||
promptOrder: [
|
||||
{ promptId: "sys", enabled: true, orderIndex: 0 },
|
||||
{ promptId: "chatHistory", enabled: true, orderIndex: 1 },
|
||||
],
|
||||
generation: {},
|
||||
unsupported: [],
|
||||
importedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
type Captured = { status: number; body: any };
|
||||
|
||||
async function call(
|
||||
pathname: string,
|
||||
init: { method?: string; body?: unknown; query?: string } = {},
|
||||
): Promise<Captured> {
|
||||
const raw = init.body === undefined ? "" : JSON.stringify(init.body);
|
||||
const req = Readable.from(raw ? [Buffer.from(raw, "utf8")] : []) as unknown as IncomingMessage;
|
||||
(req as any).method = init.method ?? "GET";
|
||||
let status = 0;
|
||||
let payload = "";
|
||||
const res = {
|
||||
writeHead(code: number) {
|
||||
status = code;
|
||||
},
|
||||
end(chunk: string) {
|
||||
payload = chunk ?? "";
|
||||
},
|
||||
} as unknown as ServerResponse;
|
||||
const handled = await handlePresetLabApi(req, res, pathname, new URLSearchParams(init.query ?? ""));
|
||||
expect(handled).toBe(true);
|
||||
return { status, body: payload ? JSON.parse(payload) : null };
|
||||
}
|
||||
|
||||
let dir = "";
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(path.join(os.tmpdir(), "preset-lab-api-"));
|
||||
process.env.WRITING_AGENT_DATA_DIR = dir;
|
||||
savePreset(preset());
|
||||
});
|
||||
afterEach(() => {
|
||||
delete process.env.WRITING_AGENT_DATA_DIR;
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe("预设工作台 API", () => {
|
||||
it("非本模块路径不接管", async () => {
|
||||
const req = Readable.from([]) as unknown as IncomingMessage;
|
||||
(req as any).method = "GET";
|
||||
const res = { writeHead() {}, end() {} } as unknown as ServerResponse;
|
||||
expect(await handlePresetLabApi(req, res, "/api/settings/x", new URLSearchParams())).toBe(false);
|
||||
});
|
||||
|
||||
it("列出预设", async () => {
|
||||
const { status, body } = await call("/api/preset-lab/presets");
|
||||
expect(status).toBe(200);
|
||||
expect(body.presets.some((p: any) => p.id === "lab")).toBe(true);
|
||||
});
|
||||
|
||||
it("读取单个预设与结构检查", async () => {
|
||||
const one = await call("/api/preset-lab/preset", { query: "presetId=lab" });
|
||||
expect(one.body.preset.id).toBe("lab");
|
||||
const checked = await call("/api/preset-lab/check", { method: "POST", body: { preset: one.body.preset } });
|
||||
expect(checked.body.issues.filter((i: any) => i.level === "error")).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("提交版本后能读回版本树", async () => {
|
||||
const committed = await call("/api/preset-lab/versions/commit", {
|
||||
method: "POST",
|
||||
body: { preset: preset(), note: "第一次" },
|
||||
});
|
||||
expect(committed.body.nodes).toHaveLength(1);
|
||||
const tree = await call("/api/preset-lab/versions", { query: "presetId=lab" });
|
||||
expect(tree.body.currentId).toBe(committed.body.nodes[0].id);
|
||||
});
|
||||
|
||||
it("保存至当前版本只覆盖节点，不新增", async () => {
|
||||
const first = await call("/api/preset-lab/versions/commit", {
|
||||
method: "POST",
|
||||
body: { preset: preset(), note: "建根" },
|
||||
});
|
||||
const nodeId = first.body.currentId;
|
||||
const edited = { ...preset(), name: "改过的名字" };
|
||||
const amended = await call("/api/preset-lab/versions/amend", {
|
||||
method: "POST",
|
||||
body: { preset: edited, note: "就地改" },
|
||||
});
|
||||
expect(amended.body.nodes).toHaveLength(1);
|
||||
expect(amended.body.currentId).toBe(nodeId);
|
||||
const node = amended.body.nodes[0];
|
||||
expect(node.id).toBe(nodeId);
|
||||
expect(node.parentId).toBe(null);
|
||||
expect(node.preset.name).toBe("改过的名字");
|
||||
expect(node.note).toBe("就地改");
|
||||
expect(typeof node.updatedAt).toBe("string");
|
||||
});
|
||||
|
||||
it("预览按用例组装消息，{{char}} 被替换", async () => {
|
||||
const { status, body } = await call("/api/preset-lab/preview", {
|
||||
method: "POST",
|
||||
body: {
|
||||
preset: preset(),
|
||||
testCase: { id: "c", name: "c", charName: "林", charDescription: "", worldBook: "", userInput: "开门", history: [] },
|
||||
},
|
||||
});
|
||||
expect(status).toBe(200);
|
||||
const text = body.messages.map((m: any) => m.content).join("\n");
|
||||
expect(text).toContain("你是林");
|
||||
expect(text).toContain("开门");
|
||||
});
|
||||
|
||||
it("测试组保存与列出", async () => {
|
||||
const saved = await call("/api/preset-lab/groups/save", {
|
||||
method: "POST",
|
||||
body: { presetId: "lab", name: "组一", cases: [{ name: "a", userInput: "hi" }] },
|
||||
});
|
||||
expect(saved.body.cases[0].id).toBeTruthy();
|
||||
const groups = await call("/api/preset-lab/groups", { query: "presetId=lab" });
|
||||
expect(groups.body.groups).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("缺少 preset 时返回 400 而不是崩溃", async () => {
|
||||
const { status, body } = await call("/api/preset-lab/check", { method: "POST", body: {} });
|
||||
expect(status).toBe(400);
|
||||
expect(body.error).toContain("preset");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,221 @@
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { Readable } from "node:stream";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import type { IncomingMessage, ServerResponse } from "node:http";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { getPreset } from "../src/preset/store.js";
|
||||
import { createNativePreset } from "../src/preset/new-preset.js";
|
||||
import type { PresetPackage } from "../src/types/preset.js";
|
||||
|
||||
vi.mock("../src/llm/client.js", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("../src/llm/client.js")>();
|
||||
return {
|
||||
...actual,
|
||||
OpenAiCompatibleProvider: class {
|
||||
async complete(messages: unknown[], options?: { caller?: string }) {
|
||||
return {
|
||||
content: `mock:${options?.caller ?? "unknown"}`,
|
||||
usage: { promptTokens: 1, completionTokens: 2, totalTokens: 3 },
|
||||
model: "mock-model",
|
||||
messages,
|
||||
};
|
||||
}
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
import { handlePresetLabApi } from "../src/server/preset-lab-handlers.js";
|
||||
|
||||
type Captured = { status: number; body: any };
|
||||
|
||||
async function call(
|
||||
pathname: string,
|
||||
init: { method?: string; body?: unknown; query?: string } = {},
|
||||
): Promise<Captured> {
|
||||
const raw = init.body === undefined ? "" : JSON.stringify(init.body);
|
||||
const req = Readable.from(raw ? [Buffer.from(raw, "utf8")] : []) as unknown as IncomingMessage;
|
||||
(req as any).method = init.method ?? "GET";
|
||||
let status = 0;
|
||||
let payload = "";
|
||||
const res = {
|
||||
writeHead(code: number) {
|
||||
status = code;
|
||||
},
|
||||
end(chunk: string) {
|
||||
payload = chunk ?? "";
|
||||
},
|
||||
} as unknown as ServerResponse;
|
||||
const handled = await handlePresetLabApi(req, res, pathname, new URLSearchParams(init.query ?? ""));
|
||||
expect(handled).toBe(true);
|
||||
return { status, body: payload ? JSON.parse(payload) : null };
|
||||
}
|
||||
|
||||
function writeProfiles(dir: string): void {
|
||||
mkdirSync(dir, { recursive: true });
|
||||
const now = new Date().toISOString();
|
||||
writeFileSync(
|
||||
path.join(dir, "profiles.json"),
|
||||
JSON.stringify({
|
||||
version: 1,
|
||||
profiles: [
|
||||
{
|
||||
id: "p1",
|
||||
name: "本地模型",
|
||||
baseUrl: "https://example.invalid/v1",
|
||||
apiKey: "test-key",
|
||||
model: "mock-model",
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
},
|
||||
],
|
||||
}),
|
||||
"utf8",
|
||||
);
|
||||
}
|
||||
|
||||
let dir = "";
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(path.join(os.tmpdir(), "preset-lab-workbench-"));
|
||||
process.env.WRITING_AGENT_DATA_DIR = dir;
|
||||
writeProfiles(dir);
|
||||
});
|
||||
afterEach(() => {
|
||||
delete process.env.WRITING_AGENT_DATA_DIR;
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
describe("工作台新建与写回", () => {
|
||||
it("读取带 BOM 的预设文件", () => {
|
||||
const presetsDir = path.join(dir, "presets");
|
||||
mkdirSync(presetsDir, { recursive: true });
|
||||
const preset = createNativePreset({ id: "bom", name: "BOM" });
|
||||
writeFileSync(path.join(presetsDir, "bom.json"), `\uFEFF${JSON.stringify(preset)}`, "utf8");
|
||||
expect(getPreset("bom")?.name).toBe("BOM");
|
||||
});
|
||||
|
||||
it("新建原生预设并生成根版本", async () => {
|
||||
const { status, body } = await call("/api/preset-lab/presets/create", {
|
||||
method: "POST",
|
||||
body: { name: "新预设", id: "fresh" },
|
||||
});
|
||||
expect(status).toBe(201);
|
||||
expect(body.preset.id).toBe("fresh");
|
||||
expect(body.preset.source).toBe("native");
|
||||
expect(body.tree.nodes).toHaveLength(1);
|
||||
expect(body.preset.prompts.some((p: any) => p.id === "chatHistory" && p.marker)).toBe(true);
|
||||
expect(body.preset.prompts.some((p: any) => p.id === "currentTurn" && p.marker)).toBe(true);
|
||||
expect(getPreset("fresh")?.name).toBe("新预设");
|
||||
});
|
||||
|
||||
it("写回前拦截结构 error，force 才落盘", async () => {
|
||||
const created = await call("/api/preset-lab/presets/create", {
|
||||
method: "POST",
|
||||
body: { name: "写回", id: "apply-me" },
|
||||
});
|
||||
const preset = created.body.preset as PresetPackage;
|
||||
preset.prompts[0]!.content = "{{setvar::a::1";
|
||||
|
||||
const blocked = await call("/api/preset-lab/preset/apply", {
|
||||
method: "POST",
|
||||
body: { preset },
|
||||
});
|
||||
expect(blocked.status).toBe(400);
|
||||
expect(blocked.body.error).toContain("结构检查");
|
||||
|
||||
const forced = await call("/api/preset-lab/preset/apply", {
|
||||
method: "POST",
|
||||
body: { preset, force: true },
|
||||
});
|
||||
expect(forced.status).toBe(200);
|
||||
expect(getPreset("apply-me")?.prompts[0]!.content).toContain("setvar");
|
||||
});
|
||||
|
||||
it("三花括号是合法的字面量花括号，不算未闭合宏", async () => {
|
||||
const created = await call("/api/preset-lab/presets/create", {
|
||||
method: "POST",
|
||||
body: { name: "三花括号", id: "triple" },
|
||||
});
|
||||
const preset = created.body.preset as PresetPackage;
|
||||
preset.prompts[0]!.content = "{{{setvar::SEMOD1::}}正文";
|
||||
const checked = await call("/api/preset-lab/check", {
|
||||
method: "POST",
|
||||
body: { preset },
|
||||
});
|
||||
expect(checked.body.issues.filter((i: any) => i.level === "error")).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("重复引用同一模块是 error", async () => {
|
||||
const created = await call("/api/preset-lab/presets/create", {
|
||||
method: "POST",
|
||||
body: { name: "重复", id: "dup-order" },
|
||||
});
|
||||
const preset = created.body.preset as PresetPackage;
|
||||
preset.promptOrder.push({ ...preset.promptOrder[0]!, orderIndex: 99 });
|
||||
const checked = await call("/api/preset-lab/check", {
|
||||
method: "POST",
|
||||
body: { preset },
|
||||
});
|
||||
expect(checked.body.issues.some((i: any) => i.level === "error" && i.message.includes("重复"))).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("草稿运行", () => {
|
||||
it("run 使用请求里的草稿而不是旧版本文件", async () => {
|
||||
const created = await call("/api/preset-lab/presets/create", {
|
||||
method: "POST",
|
||||
body: { name: "草稿", id: "draft-run" },
|
||||
});
|
||||
const preset = created.body.preset as PresetPackage;
|
||||
preset.prompts[0]!.content = "草稿规则 {{char}}";
|
||||
|
||||
const group = await call("/api/preset-lab/groups/save", {
|
||||
method: "POST",
|
||||
body: {
|
||||
presetId: preset.id,
|
||||
name: "组",
|
||||
cases: [{ name: "开门", charName: "林", userInput: "我推门" }],
|
||||
},
|
||||
});
|
||||
|
||||
const run = await call("/api/preset-lab/run", {
|
||||
method: "POST",
|
||||
body: {
|
||||
presetId: preset.id,
|
||||
groupId: group.body.id,
|
||||
profileIds: ["p1"],
|
||||
preset,
|
||||
},
|
||||
});
|
||||
|
||||
expect(run.status).toBe(200);
|
||||
expect(run.body.results).toHaveLength(1);
|
||||
expect(run.body.results[0].status).toBe("done");
|
||||
expect(run.body.results[0].caseName).toBe("开门");
|
||||
expect(run.body.results[0].content).toContain("mock:preset-test:p1");
|
||||
expect(run.body.requests[0].messages.map((m: any) => m.content).join("\n")).toContain("草稿规则 林");
|
||||
expect(run.body.requests[0].messages.map((m: any) => m.content).join("\n")).toContain("我推门");
|
||||
|
||||
const runs = await call("/api/preset-lab/runs", { query: `presetId=${preset.id}` });
|
||||
expect(runs.body.runs).toHaveLength(1);
|
||||
expect(runs.body.runs[0].id).toBe(run.body.id);
|
||||
|
||||
const one = await call("/api/preset-lab/run", {
|
||||
query: `presetId=${preset.id}&runId=${run.body.id}`,
|
||||
});
|
||||
expect(one.body.results[0].caseName).toBe("开门");
|
||||
|
||||
const marked = await call("/api/preset-lab/verdict", {
|
||||
method: "POST",
|
||||
body: {
|
||||
presetId: preset.id,
|
||||
runId: run.body.id,
|
||||
profileId: "p1",
|
||||
caseId: run.body.results[0].caseId,
|
||||
verdict: "success",
|
||||
},
|
||||
});
|
||||
expect(marked.body.results[0].verdict).toBe("success");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,177 @@
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import type { PresetPackage } from "../src/types/preset.js";
|
||||
import { buildCaseMessages } from "../src/preset/test-run.js";
|
||||
import { getTestRun, saveTestRun, updateVerdict } from "../src/preset/test-runs.js";
|
||||
import { deleteTestGroup, getTestGroup, listTestGroups, saveTestGroup } from "../src/preset/test-cases.js";
|
||||
import { deleteTestCase, listTestCases, saveTestCase } from "../src/preset/test-cases.js";
|
||||
import { loadPresetLabState, savePresetLabState } from "../src/preset/lab-state.js";
|
||||
|
||||
function preset(): PresetPackage {
|
||||
return {
|
||||
id: "pp",
|
||||
name: "pp",
|
||||
source: "native",
|
||||
prompts: [
|
||||
{ id: "sys", name: "规则", enabled: true, role: "system", content: "你是{{char}}的叙事者", marker: false, sourceIdentifier: "sys" },
|
||||
{ id: "worldInfoBefore", name: "世界书", enabled: true, role: "system", content: "", marker: true, sourceIdentifier: "worldInfoBefore" },
|
||||
{ id: "charDescription", name: "角色描述", enabled: true, role: "system", content: "", marker: true, sourceIdentifier: "charDescription" },
|
||||
{ id: "chatHistory", name: "历史", enabled: true, role: "system", content: "", marker: true, sourceIdentifier: "chatHistory" },
|
||||
{ id: "currentTurn", name: "本轮", enabled: true, role: "user", content: "", marker: true, sourceIdentifier: "currentTurn" },
|
||||
],
|
||||
promptOrder: [
|
||||
{ promptId: "sys", enabled: true, orderIndex: 0 },
|
||||
{ promptId: "worldInfoBefore", enabled: true, orderIndex: 1 },
|
||||
{ promptId: "charDescription", enabled: true, orderIndex: 2 },
|
||||
{ promptId: "chatHistory", enabled: true, orderIndex: 3 },
|
||||
{ promptId: "currentTurn", enabled: true, orderIndex: 4 },
|
||||
],
|
||||
generation: {},
|
||||
unsupported: [],
|
||||
importedAt: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
let dir = "";
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(path.join(os.tmpdir(), "preset-tests-"));
|
||||
process.env.WRITING_AGENT_DATA_DIR = dir;
|
||||
});
|
||||
afterEach(() => {
|
||||
delete process.env.WRITING_AGENT_DATA_DIR;
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe("测试用例组", () => {
|
||||
it("保存、读取、列出、删除", () => {
|
||||
const g = saveTestGroup({
|
||||
name: "基础",
|
||||
cases: [{ name: "c1", charName: "林", charDescription: "冷静", worldBook: "古镇", userInput: "进门" }],
|
||||
});
|
||||
expect(g.cases[0]!.id).toBeTruthy();
|
||||
expect(getTestGroup(g.id)?.cases[0]!.charName).toBe("林");
|
||||
expect(listTestGroups()).toHaveLength(1);
|
||||
deleteTestGroup(g.id);
|
||||
expect(listTestGroups()).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("用例（工作台的展示单位）", () => {
|
||||
it("新用例落成只装它的一份组文件，平铺列表能读回", () => {
|
||||
const saved = saveTestCase({
|
||||
case: { name: "开门", userInput: "我推开门" },
|
||||
});
|
||||
expect(saved.groupId).toBeTruthy();
|
||||
expect(saved.case.id).toBeTruthy();
|
||||
expect(listTestCases().map((item) => item.case.name)).toEqual(["开门"]);
|
||||
expect(listTestGroups()).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("用例全局共享：换预设也能读到同一套", () => {
|
||||
saveTestCase({ case: { name: "全局用例", userInput: "甲" } });
|
||||
// 没有任何 presetId 参与，读取时也不需要
|
||||
expect(listTestCases().map((item) => item.case.name)).toEqual(["全局用例"]);
|
||||
expect(listTestGroups()[0]!.presetId).toBe("global");
|
||||
});
|
||||
|
||||
it("旧版按预设分目录的用例会被并进全局目录", () => {
|
||||
const legacyDir = path.join(dir, "presets", "tests", "pp");
|
||||
mkdirSync(legacyDir, { recursive: true });
|
||||
writeFileSync(
|
||||
path.join(legacyDir, "old12345.json"),
|
||||
JSON.stringify({
|
||||
id: "old12345",
|
||||
presetId: "pp",
|
||||
name: "旧组",
|
||||
updatedAt: "2024-01-01T00:00:00.000Z",
|
||||
cases: [{ id: "c1", name: "老用例", charName: "", charDescription: "", worldBook: "", userInput: "x", history: [] }],
|
||||
}),
|
||||
"utf8",
|
||||
);
|
||||
expect(listTestCases().map((item) => item.case.name)).toEqual(["老用例"]);
|
||||
expect(getTestGroup("old12345")!.presetId).toBe("global");
|
||||
});
|
||||
|
||||
it("存回旧组里的用例时不动同组其他用例", () => {
|
||||
const group = saveTestGroup({
|
||||
name: "旧组",
|
||||
cases: [{ name: "A", userInput: "甲" }, { name: "B", userInput: "乙" }],
|
||||
});
|
||||
const target = group.cases[1]!;
|
||||
saveTestCase({
|
||||
groupId: group.id,
|
||||
case: { ...target, userInput: "乙改" },
|
||||
});
|
||||
const after = getTestGroup(group.id)!.cases;
|
||||
expect(after.map((item) => item.userInput)).toEqual(["甲", "乙改"]);
|
||||
});
|
||||
|
||||
it("删除最后一份用例时把空组文件一起收掉", () => {
|
||||
const saved = saveTestCase({ case: { name: "单条" } });
|
||||
deleteTestCase(saved.groupId, saved.case.id);
|
||||
expect(listTestCases()).toEqual([]);
|
||||
expect(listTestGroups()).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("预设的试跑状态", () => {
|
||||
it("按预设记住打开的是哪条用例，重读还在", () => {
|
||||
expect(loadPresetLabState("pp").selectedCaseId).toBe(null);
|
||||
savePresetLabState("pp", { selectedCaseId: "c1" });
|
||||
expect(loadPresetLabState("pp").selectedCaseId).toBe("c1");
|
||||
// 另一个预设不受影响
|
||||
expect(loadPresetLabState("qq").selectedCaseId).toBe(null);
|
||||
// 只传 previewCaseId 时不抹掉 selectedCaseId
|
||||
savePresetLabState("pp", { previewCaseId: "c2" });
|
||||
const state = loadPresetLabState("pp");
|
||||
expect(state.selectedCaseId).toBe("c1");
|
||||
expect(state.previewCaseId).toBe("c2");
|
||||
});
|
||||
});
|
||||
|
||||
describe("用例消息组装", () => {
|
||||
it("char 描述、世界书、历史、用户输入按顺序进入 messages，且 {{char}} 被替换", () => {
|
||||
const msgs = buildCaseMessages(preset(), {
|
||||
id: "c", name: "c", charName: "林", charDescription: "冷静的侦探", worldBook: "古镇夜雨",
|
||||
userInput: "我推门", history: [{ role: "assistant", content: "门开了。" }],
|
||||
});
|
||||
const text = msgs.map((m) => `${m.role}:${m.content}`);
|
||||
expect(text[0]).toContain("你是林的叙事者");
|
||||
expect(text.join("\n")).toContain("古镇夜雨");
|
||||
expect(text.join("\n")).toContain("冷静的侦探");
|
||||
expect(text[text.length - 1]).toBe("user:我推门");
|
||||
expect(text).toContain("assistant:门开了。");
|
||||
});
|
||||
|
||||
it("关掉 currentTurn 位置后，用户输入不再进入 messages", () => {
|
||||
const p = preset();
|
||||
const turn = p.promptOrder.find((item) => item.promptId === "currentTurn");
|
||||
if (turn) turn.enabled = false;
|
||||
const msgs = buildCaseMessages(p, {
|
||||
id: "c", name: "c", charName: "林", charDescription: "", worldBook: "",
|
||||
userInput: "这句不该出现", history: [],
|
||||
});
|
||||
expect(msgs.map((m) => m.content).join("\n")).not.toContain("这句不该出现");
|
||||
});
|
||||
|
||||
it("只替换变动项：同一预设、不同用例的 system 规则保持一致", () => {
|
||||
const a = buildCaseMessages(preset(), { id: "a", name: "a", charName: "甲", charDescription: "", worldBook: "", userInput: "x", history: [] });
|
||||
const b = buildCaseMessages(preset(), { id: "b", name: "b", charName: "甲", charDescription: "", worldBook: "", userInput: "y", history: [] });
|
||||
// 默认整形为 strict：system 会并入相邻 user，故只比较规则文本是否都在首条里\n expect(a[0]!.content.startsWith("你是甲的叙事者")).toBe(true);\n expect(b[0]!.content.startsWith("你是甲的叙事者")).toBe(true);\n expect(a[0]!.content.replace("x", "")).toBe(b[0]!.content.replace("y", ""));
|
||||
});
|
||||
});
|
||||
|
||||
describe("运行记录与手动标记", () => {
|
||||
it("保存后可以修改标记并落盘", () => {
|
||||
saveTestRun({
|
||||
id: "r1", presetId: "pp", presetVersionId: "v1", groupId: "g", createdAt: new Date().toISOString(),
|
||||
requests: [], issues: [],
|
||||
results: [{ profileId: "p", profileName: "P", model: "m", caseId: "c", status: "done", verdict: null, startedAt: "" }],
|
||||
});
|
||||
updateVerdict("pp", "r1", "p", "c", "success");
|
||||
expect(getTestRun("pp", "r1")!.results[0]!.verdict).toBe("success");
|
||||
});
|
||||
});
|
||||
|
||||
+1410
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,182 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>预设工作台 · Writing Agent</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
||||
<link
|
||||
href="https://fonts.googleapis.com/css2?family=Noto+Sans+SC:wght@400;500;600;700&family=Noto+Serif+SC:wght@500;600;700&display=swap"
|
||||
rel="stylesheet"
|
||||
/>
|
||||
<script src="/theme-init.js"></script>
|
||||
<link rel="stylesheet" href="/settings.css" />
|
||||
<link rel="stylesheet" href="/history-editor.css" />
|
||||
<link rel="stylesheet" href="/presets.css" />
|
||||
</head>
|
||||
<body class="settings-page px-page" data-preview="on" data-outline="off">
|
||||
<header class="st-top">
|
||||
<a class="st-brand" href="/">Writing Agent</a>
|
||||
<nav class="st-nav" aria-label="主导航">
|
||||
<a href="/">对话</a>
|
||||
<a href="/chat.html">直聊</a>
|
||||
<a href="/modules.html">技能</a>
|
||||
<a href="/libraries.html">库</a>
|
||||
<a href="/stats.html">Token</a>
|
||||
<a href="/presets.html" aria-current="page">预设</a>
|
||||
<a href="/settings.html">设置</a>
|
||||
</nav>
|
||||
</header>
|
||||
|
||||
<!-- 预设身份条：唯一脏状态 + 唯一主行动 -->
|
||||
<div class="px-idbar">
|
||||
<div class="px-id-main">
|
||||
<select id="preset-select" class="px-select" aria-label="选择预设"></select>
|
||||
<button type="button" id="btn-new-preset" class="px-btn px-btn-mini" title="新建预设">＋</button>
|
||||
<input
|
||||
id="preset-name"
|
||||
class="px-input px-name"
|
||||
type="text"
|
||||
aria-label="预设名称"
|
||||
placeholder="预设名称"
|
||||
/>
|
||||
<span class="px-idmeta" id="preset-meta"></span>
|
||||
<span class="px-dirty" id="dirty-pill" data-dirty="0">已同步</span>
|
||||
</div>
|
||||
<div class="px-id-actions">
|
||||
<span class="px-notice" id="notice" aria-live="polite"></span>
|
||||
<button type="button" class="px-btn" id="btn-reload" title="从磁盘重读预设文件，未保存的草稿会丢">重新载入</button>
|
||||
<button type="button" class="px-btn" id="btn-run-entry">
|
||||
<span class="px-btn-label">试跑</span><span class="px-btn-meta" id="run-entry-meta"></span>
|
||||
</button>
|
||||
<button type="button" class="px-btn" id="btn-versions" title="打开版本树">版本树</button>
|
||||
<button type="button" class="px-btn" id="btn-save-current" title="覆盖当前版本节点，不新建（Ctrl+Shift+S）">
|
||||
保存至当前版本
|
||||
</button>
|
||||
<button type="button" class="px-btn" id="btn-save-new" title="在当前节点下追加一个新版本（Ctrl+S）">
|
||||
保存为新版本
|
||||
</button>
|
||||
<button type="button" class="px-btn px-btn-primary" id="btn-apply" title="先保存为新版本，再覆盖预设文件——对话页读的是那个文件">应用到对话页</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="px-alert" id="alert" hidden></div>
|
||||
|
||||
<!-- 视图 A：编辑 -->
|
||||
<div class="px-view px-view-edit" id="view-edit">
|
||||
<nav class="px-outline" id="outline" aria-label="发送顺序"></nav>
|
||||
|
||||
<main class="px-editor" id="editor"></main>
|
||||
|
||||
<aside class="px-preview" id="preview-pane" aria-label="实际发送内容">
|
||||
<div class="px-preview-head">
|
||||
<select id="preview-case" class="px-select" aria-label="预览用例"></select>
|
||||
<button type="button" class="px-btn px-btn-mini" id="btn-preview" title="Ctrl+Enter">重算</button>
|
||||
<button type="button" class="px-btn px-btn-mini" id="btn-toggle-preview" title="显示 / 收起预览">
|
||||
收起
|
||||
</button>
|
||||
</div>
|
||||
<div class="px-band" id="issues-band" data-level="ok"></div>
|
||||
<p class="px-preview-meta" id="preview-meta"></p>
|
||||
<div class="px-messages" id="preview-out"></div>
|
||||
</aside>
|
||||
</div>
|
||||
|
||||
<!-- 视图 B：试跑 -->
|
||||
<div class="px-view px-view-run" id="view-run" hidden>
|
||||
<div class="px-runbar">
|
||||
<div class="px-runbar-row">
|
||||
<span class="px-label">当前用例</span>
|
||||
<span class="px-runbar-case" id="run-case-label">先在左侧选一个用例</span>
|
||||
<span class="px-runbar-sep" aria-hidden="true"></span>
|
||||
<span class="px-label">模型</span>
|
||||
<div class="px-picker">
|
||||
<button
|
||||
type="button"
|
||||
class="px-btn px-btn-mini px-picker-trigger"
|
||||
id="model-picker-btn"
|
||||
aria-haspopup="listbox"
|
||||
aria-expanded="false"
|
||||
>
|
||||
选择模型<span class="px-picker-count" id="model-picker-count"></span>
|
||||
</button>
|
||||
<div
|
||||
class="px-picker-menu"
|
||||
id="model-picker-menu"
|
||||
role="listbox"
|
||||
aria-multiselectable="true"
|
||||
hidden
|
||||
></div>
|
||||
</div>
|
||||
<div class="px-picked" id="picked-models" aria-label="已选模型"></div>
|
||||
<button type="button" class="px-btn px-btn-primary px-runbar-run" id="btn-run">运行此用例</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<nav class="px-caselist" id="cases" aria-label="用例" title="用例全局共用：任何预设都能拿这些用例去跑"></nav>
|
||||
|
||||
<main class="px-results-pane">
|
||||
<div class="px-results-head">
|
||||
<select id="run-select" class="px-select" aria-label="历史运行"></select>
|
||||
<button type="button" class="px-btn px-btn-mini" id="btn-refresh-runs">刷新记录</button>
|
||||
<p class="px-hint px-results-hint">不自动评分，逐条标记成功 / 失败；标记会落盘。</p>
|
||||
</div>
|
||||
<div class="px-results" id="results"></div>
|
||||
</main>
|
||||
</div>
|
||||
|
||||
<!-- 版本抽屉（低频，从身份条进入） -->
|
||||
<div class="px-scrim" id="scrim" hidden></div>
|
||||
<aside class="px-drawer" id="version-drawer" hidden aria-label="版本">
|
||||
<div class="px-drawer-head">
|
||||
<h2>版本</h2>
|
||||
<button type="button" class="px-btn px-btn-mini" id="btn-close-versions">关闭</button>
|
||||
</div>
|
||||
<div class="px-drawer-body">
|
||||
<div id="version-tree" role="tree"></div>
|
||||
<p class="px-hint" id="version-hint">还没有版本。</p>
|
||||
<label class="px-field">
|
||||
<span>本次修改说明</span>
|
||||
<input type="text" id="note-input" placeholder="例：收紧叙事视角" />
|
||||
</label>
|
||||
<div class="px-row">
|
||||
<button type="button" class="px-btn px-btn-primary" id="btn-save-version">保存为新版本</button>
|
||||
<button type="button" class="px-btn" id="btn-save-version-current">保存至当前版本</button>
|
||||
</div>
|
||||
<div class="px-row">
|
||||
<button type="button" class="px-btn" id="btn-load-version">载入选中</button>
|
||||
<button type="button" class="px-btn" id="btn-fork-version">从选中分叉</button>
|
||||
</div>
|
||||
<p class="px-hint" id="dirty-hint"></p>
|
||||
<p class="px-hint">
|
||||
点节点弹菜单：「读取」换进编辑器，「从它分叉」用当前草稿在它下面开新分支。
|
||||
圆点是当前编辑位置，分叉会另开一条线。
|
||||
</p>
|
||||
<p class="px-hint">
|
||||
「保存为新版本」（Ctrl+S）在当前节点下追加子节点；「保存至当前版本」直接覆盖当前节点。
|
||||
「应用到对话页」会先保存为新版本，再覆盖 <code>presets/<id>.json</code>——对话页读的是那个文件。
|
||||
「重新载入」从磁盘重读预设文件，未保存的草稿会丢。
|
||||
</p>
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
<!-- 用例浮窗（字段多、还要写历史，用大浮窗而不是窄侧栏） -->
|
||||
<section
|
||||
class="px-modal"
|
||||
id="case-modal"
|
||||
role="dialog"
|
||||
aria-modal="true"
|
||||
aria-labelledby="case-drawer-title"
|
||||
hidden
|
||||
>
|
||||
<div class="px-modal-head">
|
||||
<h2 id="case-drawer-title">用例</h2>
|
||||
<button type="button" class="px-btn px-btn-mini" id="btn-close-case">完成</button>
|
||||
</div>
|
||||
<div class="px-modal-body" id="case-form"></div>
|
||||
</section>
|
||||
|
||||
<script type="module" src="/presets.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
+2455
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user