测试:轻量拓展分析 NL 触发两轮测试交付(单测 79 + 集成 23 + 前端契约 20)
This commit is contained in:
@@ -0,0 +1,98 @@
|
|||||||
|
# 轻量版拓展分析自然语言触发 — 第一轮测试报告
|
||||||
|
|
||||||
|
- 时间:2026-08-27
|
||||||
|
- 被测提交:a60f45754(轻量拓展分析自然语言触发,4 文件 +813/-8,未 push)
|
||||||
|
- 测试环境:/home/xzq/miniconda3/envs/plantai-bff/bin/python(Python 3.11.15 / pytest 9.1.1 / pytest-cov 7.1.0;本轮补充安装 pyyaml/openai/pandas/requests/openpyxl 后方可跑通 main.py 相关测试)
|
||||||
|
|
||||||
|
## 测试文件清单
|
||||||
|
|
||||||
|
| 文件 | 新增/追加 | 用例数 | 结果 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| tests/unit/test_extension_stage_lightweight_nl_intent.py | 新增 | 58 | 全绿 |
|
||||||
|
| tests/unit/test_extension_stage_lightweight_nl_backend.py | 新增 | 18 | 全绿 |
|
||||||
|
| tests/unit/test_extension_stage_lightweight_frontend_contract.py | 追加 5 个测试(未删任何现有断言,原 15 → 现 20) | 20 | 全绿 |
|
||||||
|
|
||||||
|
## 通过/失败数
|
||||||
|
|
||||||
|
| 组 | 结果 |
|
||||||
|
|---|---|
|
||||||
|
| A 组(nl_intent 单元) | 58 passed |
|
||||||
|
| B 组(main.py 后端集成) | 18 passed |
|
||||||
|
| C 组(前端契约) | 20 passed |
|
||||||
|
| 新增三件套合计 | 96 passed |
|
||||||
|
| 回归:Phase A 三件套(agent/input_binding/golden_runner_semantic) | 6 passed |
|
||||||
|
| 回归:test_agent_chat_actions + deepseek_semantics | 46 passed |
|
||||||
|
| 回归:全套 test_extension_stage_lightweight*.py | 312 passed + **6 既有失败** + 1 skip |
|
||||||
|
|
||||||
|
**6 个既有失败**(已在父提交 90f51ed60 干净 worktree 上逐一复现,与本改动无关,不拦路):
|
||||||
|
1. test_extension_stage_lightweight_fact_contract.py::test_build_execution_facts_keeps_unmatched_outputs_without_guessing
|
||||||
|
2. test_extension_stage_lightweight_runtime_binding.py::test_shell_patch_targets_record_hash_literal_kind_and_exact_count
|
||||||
|
3. test_extension_stage_lightweight_runtime_binding.py::test_same_uploaded_path_in_two_steps_forms_one_safe_path_binding
|
||||||
|
4. test_extension_stage_lightweight_runtime_binding.py::test_repeated_parameter_literal_across_facts_is_not_patchable
|
||||||
|
5. test_extension_stage_lightweight_runtime_package.py::test_unrelated_implicit_output_remains_blocked
|
||||||
|
6. test_extension_stage_lightweight_target_launchers.py::test_target_launchers_share_lightweight_extension_runtime_roots
|
||||||
|
|
||||||
|
## 覆盖率
|
||||||
|
|
||||||
|
`python3 -m pytest --cov=gfa_core.extension_stage_lightweight.nl_intent --cov-report=term-missing tests/unit/test_extension_stage_lightweight_nl_intent.py`
|
||||||
|
|
||||||
|
- gfa_core/extension_stage_lightweight/nl_intent.py:**187 stmts / 0 miss / 100%**
|
||||||
|
- 无未覆盖行(补测过程:93% → 98% → 99% → 100%,最终 0 遗漏)
|
||||||
|
|
||||||
|
## 失败用例+关键报错
|
||||||
|
|
||||||
|
本轮无功能缺陷导致的测试失败(测试自身两处断言问题已修复:集合类型断言、reason 前缀断言)。
|
||||||
|
|
||||||
|
## 重点验证结论(对应 developer 标记风险 R1-R4 + 测试要求 1-10)
|
||||||
|
|
||||||
|
### R1 目录截断(P0 缺陷确认)
|
||||||
|
真实 store 实测:61 个 published 记录,`_build_catalog` 只放前 11 个即达 11276 字符,truncated=True。
|
||||||
|
「拓展分析」强触发词下 `_rough_candidates` 返回全部 61 个,但目录按目录名排序截断后 **12_/13_/14_/15_/16_/17_/18_ 共 50 个扩展(含 manifestqc、tfbs、kaks_cis、motif、pav_cnv 等 18 案例中的后 7 个)LLM 完全不可见**,任何消息都无法命中这些扩展。
|
||||||
|
测试固化:test_real_store_catalog_truncation_smoke、test_build_catalog_truncates_when_over_max_chars。
|
||||||
|
|
||||||
|
### R2 中文关键词粗筛(缺陷确认)
|
||||||
|
真实 store 实测:「帮我做样本表校验」→ 0 候选、不调 LLM(function_summary 全英文,纯中文功能词无法命中 `_summary_keywords` 英文 token 与中文子串)。只有强触发词(拓展分析/扩展分析/extension)或工具词(hmmsearch/hmmer/blast/blastp)才能进 LLM。**注意:工具词路径会把全部 61 个记录都当候选(无法按工具区分),再叠加 R1 截断,实际能解析的只有前 11 个。**
|
||||||
|
测试固化:test_real_store_pure_chinese_function_words_miss_rough_candidates。
|
||||||
|
|
||||||
|
### R3 prefill 字段位置(实现一致)
|
||||||
|
已验证:`LightweightIntentResolution.prefill` 在 dataclass 顶层;`_match_lightweight_extension_intent` 返回 dict 中 `params.prefill`(顶层副本)与 `params.llm_slots`;前端 `openLightweightNlRunDialog` 读 `intentParams.llm_slots` 的 input_overrides/parameter_overrides/explicit_prefix,不依赖顶层。C 组断言按实际实现写,无错位。
|
||||||
|
|
||||||
|
### R4 extension_name 英文(产品缺陷候选,记录待验收)
|
||||||
|
已确认:`extension_name` 取 function_summary 英文首句(≤60 字符),后端 message 与前端向导标题会显示英文(如 "This tool searches sequences with hmmsearch.")。测试固化:test_resolution_uses_first_sentence_as_extension_name。建议后续加中文 display_name 映射。
|
||||||
|
|
||||||
|
### 测试要求 1-10 覆盖情况
|
||||||
|
1. 无触发词 → extension_id None 且 client_factory 计数为 0(参数化 3 条消息)✅
|
||||||
|
2. 触发词 + 无 records → 不调 LLM(reason=no_published_records)✅
|
||||||
|
3. _build_catalog:只含 replaceable inputs 的 {path,role,description}、剔 locked input、parameter 无 value、含 function_summary、超长截断置 truncated ✅
|
||||||
|
4. _sanitize_resolution:编造 path/fact_id/extension_id/非 str value/confidence 非法 → None+原因(参数化 7 种非法输入 + 4 种非对象);合法 → 正确映射 ✅
|
||||||
|
5. LLM 抛异常/坏 JSON/缺 choices → None resolution 不抛(reason=llm_unavailable_or_invalid 或 invalid_resolution: 前缀)✅
|
||||||
|
6. happy path:「使用 HMM 拓展分析对刚上传的文件进行分析,参数 e-value=1e-5」→ input_overrides/parameter_overrides/confidence/prefill 全对 ✅
|
||||||
|
7. load_published_lightweight_records:published/confirmation_pending/baseline_failed/坏 JSON/非对象 → 只返回 published;坏 store 根返回 [] ✅
|
||||||
|
8. job_file_hints 进入 LLM user 消息 JSON(并测 MAX_JOB_FILE_HINTS 截断 200)✅
|
||||||
|
9. 参数值约束:sanitize 只做结构校验(fact_id 合法即收),不校验"参数是否在用户消息中出现"(prompt 层约束),测试固化该行为 ✅
|
||||||
|
10. 覆盖率 100%(见上)✅
|
||||||
|
|
||||||
|
### B 组关键用例
|
||||||
|
- _match_lightweight_extension_intent:命中返回 lightweight:True/llm_slots/extension_store/prefill/confidence/parse_reason;空 store、resolution None、extension_id None → 均返回 {} ✅
|
||||||
|
- 回退三处:Phase A 命中不标 lightweight;Phase A 空列表 → 回退轻量(三个空返回点中 candidates 为空与 trigger 为空两处直接测到;rnaseq 短路点静态确认回退存在,未单独构造 rnaseq 消息,因为该消息本身也不命中轻量粗筛)✅
|
||||||
|
- _lightweight_job_file_hints:input+workspace 相对路径、空 job_id → []、200 上限 ✅
|
||||||
|
- import-paths 端点:合法 /home/xzq/ 下文件复制进 _nl_imports、同名冲突加 _1 后缀、前缀外路径 400、不存在路径 400、空 job_id 400 ✅(注:source 必须在 /home/xzq/ 下构造,tmp_path 在 /tmp 会被前缀校验拒绝)
|
||||||
|
- _run_agent_chat 短路:预闸命中 + 轻量 intent → action="propose_extension_stage"、model_used="rule",且 monkeypatch 证明未触碰 LLMManager/RouterAgent(会抛异常);预闸不命中 → 走正常 MainAgent 流程 ✅
|
||||||
|
- /api/agent/plan 守卫端到端:轻量 intent 不调 _extension_stage_plan_response;Phase A 命中正常调用 ✅
|
||||||
|
|
||||||
|
## 遗留问题清单
|
||||||
|
|
||||||
|
| 编号 | 位置 | 现象 | 建议修复 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1(P0) | gfa_core/extension_stage_lightweight/nl_intent.py `_build_catalog`(约 128-173 行) | 真实 store 61 个 published 记录只进前 11 个即截断(11276/12000 字符),目录按目录名排序,12_ 及以后的 50 个扩展(含 18 案例中 12-18 的 7 类)LLM 不可见,「拓展分析」强触发词下也永远无法命中 | 截断策略需改为"全部进目录、按相关性排序"或"先粗筛再目录化"(按关键词把候选缩到相关子集),至少保证 18 案例的功能词能进目录;12_ 以后的扩展在截断前应有兜底(如只放 extension_id+summary 的轻量条目) |
|
||||||
|
| 2(P1) | 同文件 `_rough_candidates`(约 108-122 行) | 纯中文功能词(如"样本表校验")0 候选不调 LLM,只能靠强触发词或英文工具词进入;工具词 hmmsearch/hmmer/blast 路径会把全部 61 个记录都当候选(不按工具区分) | 中文 summary 子串匹配对 18 案例 prompt 影响需产品确认;工具词路径建议按工具名映射到对应扩展(如 hmmsearch → 03_ 系列),或至少接受"全候选+目录排序优化" |
|
||||||
|
| 3(P1) | gfa_core/extension_stage_lightweight/nl_intent.py `_rough_candidates`(约 113-114 行) | `str(record.get("extension_id"))` 对缺失 id 的记录产出 "None" 字符串候选 | 改为 `record.get("extension_id") and str(...)` 过滤,或直接 `[r["extension_id"] for r in records if r.get("extension_id")]` |
|
||||||
|
| 4(P2) | 同文件 `resolve_lightweight_extension_intent`(约 265-346 行) | `extension_name` 取 function_summary 英文首句(≤60),后端 message 与前端向导标题显示英文 | 产品验收时确认是否需中文 display_name 映射;若需,在 record 或 catalog 层加中文名字段 |
|
||||||
|
| 5(P2) | apps/web/vue3/main.py `nl_import_paths`(约 4449-4512 行) | 目录导入用 `shutil.copytree(dirs_exist_ok=True)` 合并进目标目录,且同名目录无冲突后缀(与文件路径的 _1 后缀策略不一致);多路径循环中前一个成功、后一个 400 时已复制的文件不回滚 | 目录同名冲突建议同样加后缀;整体事务性按需评估(低风险,向导场景可接受部分成功) |
|
||||||
|
| 6(P3/信息) | 全套 lightweight 回归 | 6 个既有失败(见上清单),父提交 90f51ed60 复现,与本改动无关 | 不拦路;建议另开任务修复 |
|
||||||
|
| 7(P3/信息) | 测试环境 | plantai-bff 环境缺 pyyaml/openai/pandas/requests/openpyxl,main.py 相关测试此前跑不通(test_agent_chat_actions 修复前 27 failed) | 已补充安装;建议把依赖清单写进仓库 requirements 或测试文档 |
|
||||||
|
|
||||||
|
## 测试环境备注
|
||||||
|
- 本轮在 plantai-bff conda 环境补充安装了:pyyaml、openai、pandas、requests、openpyxl(均为项目既有依赖,纯 pip 安装,未动仓库文件)。
|
||||||
|
- 验证既有失败时在父提交 90f51ed60 建了临时 worktree,验证后已删除(git worktree remove --force)。
|
||||||
|
- 工作区改动仅限测试文件:tests/unit/test_extension_stage_lightweight_nl_intent.py(新增)、tests/unit/test_extension_stage_lightweight_nl_backend.py(新增)、tests/unit/test_extension_stage_lightweight_frontend_contract.py(追加 5 个测试,现有断言未删)。开发代码未触碰,未 push。
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# 轻量版拓展分析自然语言触发 — 第二轮测试报告
|
||||||
|
|
||||||
|
- 时间:2026-08-27
|
||||||
|
- 被测提交:db610031c + afdb1614a(修复第一轮 7 项反馈 + PM 裁断修正,均未 push)
|
||||||
|
- 测试环境:/home/xzq/miniconda3/envs/plantai-bff/bin/python(Python 3.11.15 / pytest 9.1.1 / pytest-cov 7.1.0)
|
||||||
|
|
||||||
|
## 测试文件清单
|
||||||
|
|
||||||
|
| 文件 | 状态 | 用例数 | 结果 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| tests/unit/test_extension_stage_lightweight_nl_intent.py | 更新+新增 | 79(上轮 58) | 全绿 |
|
||||||
|
| tests/unit/test_extension_stage_lightweight_nl_backend.py | 更新+新增 | 23(上轮 18) | 全绿 |
|
||||||
|
| tests/unit/test_extension_stage_lightweight_frontend_contract.py | 不变(与新实现一致,未删断言) | 20 | 全绿 |
|
||||||
|
|
||||||
|
## 通过/失败数
|
||||||
|
|
||||||
|
| 组 | 结果 |
|
||||||
|
|---|---|
|
||||||
|
| A 组(nl_intent 单元) | 79 passed,覆盖率 100%(223 stmts / 0 miss) |
|
||||||
|
| B 组(main.py 后端集成) | 23 passed |
|
||||||
|
| C 组(前端契约) | 20 passed |
|
||||||
|
| 相关四文件回归(A+B+C+deepseek_semantics) | 131 passed |
|
||||||
|
| Phase A 三件套(agent/input_binding/golden_runner_semantic) | 6 passed |
|
||||||
|
| test_agent_chat_actions | 36 passed |
|
||||||
|
| 全套 test_extension_stage_lightweight*.py | 337 passed + 6 既有失败(与第一轮相同 6 个,父提交复现,不拦路)+ 1 skip |
|
||||||
|
|
||||||
|
## 覆盖率
|
||||||
|
|
||||||
|
`nl_intent.py`:**223 stmts / 0 miss / 100%**。第一轮后新增代码(union 粗筛、_extension_id_tokens、_parameter_flag_tokens、_record_keyword_sources、compact_extension_name、catalog 24000、truncated reason、resolve 排序)全部覆盖。补测过程中 99%→100% 的一次补齐为 `_parameter_flag_tokens` 非 Mapping 元素跳过分支。
|
||||||
|
|
||||||
|
## 失败用例+关键报错
|
||||||
|
|
||||||
|
本轮无功能缺陷导致的失败;5 个上轮用例因行为有意变更而更新(工具词不再全量、extension_name 改紧凑形式、blast/hmmer 用例 fixture 隔离)。
|
||||||
|
|
||||||
|
## 第二轮修复验证结论(对应 main 要求的 5 项)
|
||||||
|
|
||||||
|
### 1. union 粗筛策略(afdb1614a)— 通过
|
||||||
|
- 带工具词+强触发词 → 关键词子集(不因强触发词全量):test_rough_candidates_union_strategy_with_strong_trigger_and_keyword ✅
|
||||||
|
- 纯强触发词无关键词命中 → 全量兜底:test_rough_candidates_pure_strong_trigger_falls_back_to_all ✅
|
||||||
|
- 纯工具词无强触发词 → 子集:参数化用例 "用 hmmsearch 搜索结构域" → ["ext_hmm"] ✅
|
||||||
|
- 无任何命中 → []:test_rough_candidates_no_match_returns_empty ✅
|
||||||
|
- 命中数降序(同分按 id):test_rough_candidates_sorted_by_hits_desc ✅
|
||||||
|
- 缺失 extension_id 过滤不产出 None:test_rough_candidates_skips_missing_extension_id ✅
|
||||||
|
- resolve 按候选分排序目录:test_resolve_orders_records_by_candidate_score ✅
|
||||||
|
|
||||||
|
### 2. compact_extension_name — 通过
|
||||||
|
参数化 6 例:去尾缀时间戳(`03___hmm_1787130993548`→`03___hmm`)、超长截 30、空串、无时间戳原样 ✅
|
||||||
|
|
||||||
|
### 3. catalog 24000 上限与 truncated reason — 通过
|
||||||
|
- MAX_CATALOG_CHARS == 24000 断言 ✅
|
||||||
|
- 60 条长 summary 记录截断、总长 ≤ 上限、截断点后首条追加超限 ✅
|
||||||
|
- 去重 seen_ids ✅
|
||||||
|
- 截断时 reason="ok; 候选过多已截断,请补充工具名" ✅
|
||||||
|
|
||||||
|
### 4. _extension_id_tokens / _parameter_flag_tokens — 通过
|
||||||
|
- id token:`03___hmm_1787130993548` → 含 hmm、剔 len<3 ✅
|
||||||
|
- flag token:`--samples` 保留、单字母剔除、停用词剔除、无 contract 返回 []、非 Mapping 元素跳过 ✅
|
||||||
|
- _record_keyword_sources 三源合并(summary + id token + flag)✅
|
||||||
|
|
||||||
|
### 5. B 组 message 中文模板 — 通过
|
||||||
|
- extension_id 为主、summary 首句截 60 字括号补充:"已识别为轻量拓展分析:ext_hmm(This tool searches sequences with hmmsearch)。请按引导上传或选择输入文件并核对参数后执行。" ✅
|
||||||
|
- 无 summary 时无括号补充 ✅
|
||||||
|
- params 新增 function_summary 字段 ✅
|
||||||
|
|
||||||
|
### 6. import-paths 第二轮修复 — 通过
|
||||||
|
- 目录同名冲突 _1 后缀(原目录不被覆盖)✅
|
||||||
|
- 多路径先整体校验后复制:合法+非法混合 → 400 且合法文件未复制 ✅
|
||||||
|
- 不存在路径混入 → 400 且未复制 ✅
|
||||||
|
- 多合法路径全部复制 ✅
|
||||||
|
|
||||||
|
### 7. PM 实测对齐(真实 store 固化)— 通过
|
||||||
|
test_real_store_pm_alignment_counts:HMM→9、DIAMOND→8、OrthoFinder→3、KaKs→2、ManifestQC→5、纯拓展分析→61,与 PM 实测完全一致;且 HMM 子集(9 条)构建 catalog 不截断(truncated=False),P0 截断问题在带工具词场景已解决。
|
||||||
|
|
||||||
|
### 8. C 组一致性 — 通过
|
||||||
|
index.html 两提交间零改动,现有 20 个契约测试(含上轮追加 5 个)全部保持绿色,无需调整。
|
||||||
|
|
||||||
|
## 遗留问题清单
|
||||||
|
|
||||||
|
| 编号 | 位置 | 现象 | 建议修复 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1(P2) | nl_intent.py `_rough_candidates` / `_summary_keywords` | 纯中文功能词(如「样本表校验」)仍 0 候选不调 LLM——manifestqc 等中文场景只能靠「拓展分析」强触发词全量兜底(61 候选 + 24000 截断后 13 条进 catalog,排序靠目录名,manifestqc 18_ 系列仍在截断线外)。实测「用 ManifestQC 拓展分析」走 id token 命中可解,但用户不知工具英文名的场景仍受限 | 产品层面确认是否需要中文功能词→工具映射表;或截断时按 18 案例优先级排序 |
|
||||||
|
| 2(P3) | nl_intent.py `_extension_id_tokens` | extension_id 的通用英文 token(如 "ext"、数字时间戳)也进关键词源,实测 "ext" 已在 _ENGLISH_STOPWORDS 之外(是 "extension" 的子串场景会误命中);上轮实测 blast 消息误命中 "against" 属 summary 普通词干扰,union 策略下仍可能存在这类弱命中把无关记录顶进子集 | 低风险;可考虑给关键词源加权重(id token > summary > flag),弱命中不参与排序竞争 |
|
||||||
|
| 3(P3) | 全套 lightweight 回归 | 6 个既有失败(与第一轮相同清单),父提交 90f51ed60 复现,与本改动无关 | 不拦路;建议另开任务 |
|
||||||
|
|
||||||
|
## 测试环境备注
|
||||||
|
- 环境同第一轮(plantai-bff + 已补装的 pyyaml/openai/pandas/requests/openpyxl)。
|
||||||
|
- 工作区改动仅测试文件:tests/unit/test_extension_stage_lightweight_nl_intent.py(更新+新增)、tests/unit/test_extension_stage_lightweight_nl_backend.py(更新+新增)。开发代码未触碰,未 push。
|
||||||
@@ -248,3 +248,55 @@ assert.strictEqual(parameterRows.find((row) => row.item === '可改参数').valu
|
|||||||
intent = source[source.index("const runExtensionStageFromIntent"):source.index("const extensionOutputIcon")]
|
intent = source[source.index("const runExtensionStageFromIntent"):source.index("const extensionOutputIcon")]
|
||||||
assert "if (isLightweightExtensionId(extensionId))" in intent
|
assert "if (isLightweightExtensionId(extensionId))" in intent
|
||||||
assert intent.index("openLightweightRuntimeCard(extensionId)") < intent.index("executeExtensionStageRun(")
|
assert intent.index("openLightweightRuntimeCard(extensionId)") < intent.index("executeExtensionStageRun(")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 自然语言触发(轻量拓展分析)追加断言 ----------
|
||||||
|
|
||||||
|
def test_lightweight_nl_run_wizard_and_state_exist_in_source() -> None:
|
||||||
|
source = _source()
|
||||||
|
|
||||||
|
assert "lightweightNlRun" in source
|
||||||
|
assert "const lightweightNlRun = reactive({" in source
|
||||||
|
assert "let lightweightNlRunResolve = null;" in source
|
||||||
|
assert "openLightweightNlRunDialog" in source
|
||||||
|
assert "executeLightweightNlRun" in source
|
||||||
|
assert "handleNlRunFileDrop" in source
|
||||||
|
assert "importNlRunServerPaths" in source
|
||||||
|
assert "cancelLightweightNlRun" in source
|
||||||
|
assert "confirmLightweightNlRun" in source
|
||||||
|
assert "nextLightweightNlRunStep" in source
|
||||||
|
|
||||||
|
|
||||||
|
def test_lightweight_nl_run_uploads_with_progress_and_imports_server_paths() -> None:
|
||||||
|
source = _source()
|
||||||
|
|
||||||
|
assert "xhr.upload.onprogress" in source
|
||||||
|
assert "el-progress" in source
|
||||||
|
assert "/api/nl-import-paths" in source
|
||||||
|
assert "input/" in source
|
||||||
|
|
||||||
|
|
||||||
|
def test_lightweight_nl_run_prefills_from_llm_slots() -> None:
|
||||||
|
source = _source()
|
||||||
|
|
||||||
|
assert "llm_slots" in source
|
||||||
|
assert "input_overrides" in source
|
||||||
|
assert "parameter_overrides" in source
|
||||||
|
assert "explicit_prefix" in source
|
||||||
|
assert "parse_reason" in source
|
||||||
|
|
||||||
|
|
||||||
|
def test_lightweight_nl_run_intent_context_and_conversation_replies() -> None:
|
||||||
|
source = _source()
|
||||||
|
|
||||||
|
assert "intent_context" in source
|
||||||
|
assert "fromIntent" in source
|
||||||
|
assert "addAgentMessage" in source
|
||||||
|
assert "taskType === 'extension_stage'" in source
|
||||||
|
|
||||||
|
|
||||||
|
def test_lightweight_nl_run_prefill_flag_wired() -> None:
|
||||||
|
source = _source()
|
||||||
|
|
||||||
|
assert "prefill" in source
|
||||||
|
assert "已识别为轻量拓展分析" not in source # 消息文案由后端生成,前端按 taskType 路由
|
||||||
|
|||||||
@@ -0,0 +1,562 @@
|
|||||||
|
"""轻量拓展分析自然语言触发的后端集成测试(main.py 侧)。
|
||||||
|
|
||||||
|
monkeypatch BBAV5_LIGHTWEIGHT_EXTENSION_STORE 指向 tmp store,
|
||||||
|
monkeypatch nl_intent.resolve_lightweight_extension_intent 返回固定 resolution。
|
||||||
|
风格对齐 tests/unit/test_agent_chat_actions.py。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from apps.web.vue3 import main as WEB_MAIN
|
||||||
|
from gfa_core.extension_stage_lightweight import nl_intent
|
||||||
|
|
||||||
|
STORE_ENV = "BBAV5_LIGHTWEIGHT_EXTENSION_STORE"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- fixtures / helpers ----------
|
||||||
|
|
||||||
|
def _write(root: Path, relative: str, content: str) -> Path:
|
||||||
|
path = root / relative
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(content, encoding="utf-8")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def _record(extension_id: str) -> dict:
|
||||||
|
return {
|
||||||
|
"extension_id": extension_id,
|
||||||
|
"status": "published",
|
||||||
|
"semantic_candidate": {"function_summary": "This tool searches sequences with hmmsearch."},
|
||||||
|
"runtime_binding_contract": {
|
||||||
|
"inputs": [
|
||||||
|
{"path": "family.hmm", "role": "HMM profile input", "description": "profile", "replaceable": True},
|
||||||
|
{"path": "01.col.primary.pep.fa", "role": "Protein sequence input", "description": "seqs", "replaceable": True},
|
||||||
|
],
|
||||||
|
"parameters": [
|
||||||
|
{"fact_id": "step_002:parameter:003", "flag": "-E", "display_name": "E-value threshold", "description": "evalue"},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _write_published_record(store_root: Path, extension_id: str) -> None:
|
||||||
|
_write(store_root, f"extensions/{extension_id}/extension.json", json.dumps(_record(extension_id)))
|
||||||
|
|
||||||
|
|
||||||
|
def _resolution(extension_id: str = "ext_hmm") -> nl_intent.LightweightIntentResolution:
|
||||||
|
return nl_intent.LightweightIntentResolution(
|
||||||
|
extension_id=extension_id,
|
||||||
|
extension_name="This tool searches sequences with hmmsearch.",
|
||||||
|
input_overrides={"family.hmm": "input/job-nl-1/_nl_imports/family.hmm"},
|
||||||
|
parameter_overrides={"step_002:parameter:003": "1e-5"},
|
||||||
|
explicit_prefix="outputs/nl",
|
||||||
|
confidence=0.9,
|
||||||
|
prefill=True,
|
||||||
|
reason="ok",
|
||||||
|
candidates_considered=[extension_id],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _patch_resolver(monkeypatch, resolution):
|
||||||
|
monkeypatch.setattr(
|
||||||
|
nl_intent,
|
||||||
|
"resolve_lightweight_extension_intent",
|
||||||
|
lambda message, records, environ=None, client_factory=None, job_file_hints=None: resolution,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def lightweight_store(monkeypatch, tmp_path: Path) -> Path:
|
||||||
|
store_root = tmp_path / "lw-store"
|
||||||
|
monkeypatch.setenv(STORE_ENV, str(store_root))
|
||||||
|
return store_root
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 1. _match_lightweight_extension_intent ----------
|
||||||
|
|
||||||
|
def test_match_lightweight_intent_returns_full_contract(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
_patch_resolver(monkeypatch, _resolution())
|
||||||
|
|
||||||
|
intent = WEB_MAIN._match_lightweight_extension_intent("使用 HMM 拓展分析", "job-nl-1")
|
||||||
|
|
||||||
|
assert intent.get("lightweight") is True
|
||||||
|
assert intent["stage"] == "extension_stage"
|
||||||
|
params = intent["params"]
|
||||||
|
assert params["extension_id"] == "ext_hmm"
|
||||||
|
assert params["extensionName"] == "This tool searches sequences with hmmsearch."
|
||||||
|
assert params["extension_store"] == "lightweight"
|
||||||
|
assert params["llm_slots"] == {
|
||||||
|
"input_overrides": {"family.hmm": "input/job-nl-1/_nl_imports/family.hmm"},
|
||||||
|
"parameter_overrides": {"step_002:parameter:003": "1e-5"},
|
||||||
|
"explicit_prefix": "outputs/nl",
|
||||||
|
}
|
||||||
|
assert params["prefill"] is True
|
||||||
|
assert params["confidence"] == 0.9
|
||||||
|
assert params["parse_reason"] == "ok"
|
||||||
|
assert intent["confidence"] == 0.9
|
||||||
|
# message 中文模板:extension_id 为主、summary 首句截 60 字括号补充
|
||||||
|
assert "已识别为轻量拓展分析" in intent["message"]
|
||||||
|
assert "ext_hmm" in intent["message"]
|
||||||
|
assert "This tool searches sequences with hmmsearch" in intent["message"]
|
||||||
|
assert "请按引导上传或选择输入文件并核对参数后执行" in intent["message"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_match_lightweight_intent_empty_store_returns_empty_dict(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
# store 无任何 published record
|
||||||
|
intent = WEB_MAIN._match_lightweight_extension_intent("使用拓展分析", "job-x")
|
||||||
|
assert intent == {}
|
||||||
|
|
||||||
|
|
||||||
|
def test_match_lightweight_intent_none_resolution_returns_empty_dict(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
_patch_resolver(monkeypatch, None) # resolution None(模拟 LLM 降级)
|
||||||
|
|
||||||
|
intent = WEB_MAIN._match_lightweight_extension_intent("使用拓展分析", "job-x")
|
||||||
|
assert intent == {}
|
||||||
|
|
||||||
|
|
||||||
|
def test_match_lightweight_intent_none_extension_id_returns_empty_dict(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
resolution = _resolution()
|
||||||
|
resolution = nl_intent.LightweightIntentResolution(
|
||||||
|
extension_id=None,
|
||||||
|
extension_name=None,
|
||||||
|
input_overrides={},
|
||||||
|
parameter_overrides={},
|
||||||
|
explicit_prefix=None,
|
||||||
|
confidence=0.0,
|
||||||
|
prefill=False,
|
||||||
|
reason="no_trigger",
|
||||||
|
candidates_considered=[],
|
||||||
|
)
|
||||||
|
_patch_resolver(monkeypatch, resolution)
|
||||||
|
|
||||||
|
intent = WEB_MAIN._match_lightweight_extension_intent("使用拓展分析", "job-x")
|
||||||
|
assert intent == {}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 2. _match_extension_stage_intent 回退 ----------
|
||||||
|
|
||||||
|
def _phase_a_ext(extension_id: str = "legacy_ext", status: str = "active", nl_triggers=None) -> dict:
|
||||||
|
return {
|
||||||
|
"extension_id": extension_id,
|
||||||
|
"name": extension_id,
|
||||||
|
"display_name": f"{extension_id} display",
|
||||||
|
"status": status,
|
||||||
|
"nl_triggers": nl_triggers or ["hmmsearch"],
|
||||||
|
"requirements": {"system": ["hmmsearch"]},
|
||||||
|
"analysis_type": "hmmer_search",
|
||||||
|
"parameters": {},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_extension_stage_intent_keeps_phase_a_when_published(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
"""Phase A 命中(构造 nl_triggers 命中)时不回退轻量匹配。"""
|
||||||
|
monkeypatch.setattr(
|
||||||
|
WEB_MAIN.extension_stage_agent,
|
||||||
|
"list_extensions",
|
||||||
|
lambda include_drafts=False: [_phase_a_ext()],
|
||||||
|
)
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
_patch_resolver(monkeypatch, _resolution())
|
||||||
|
|
||||||
|
intent = WEB_MAIN._match_extension_stage_intent("使用 hmmsearch 做结构域分析", "job-x")
|
||||||
|
|
||||||
|
assert intent.get("lightweight") is None
|
||||||
|
assert intent.get("params", {}).get("extension_id") == "legacy_ext"
|
||||||
|
|
||||||
|
|
||||||
|
def test_extension_stage_intent_falls_back_to_lightweight_when_phase_a_empty(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(
|
||||||
|
WEB_MAIN.extension_stage_agent,
|
||||||
|
"list_extensions",
|
||||||
|
lambda include_drafts=False: [],
|
||||||
|
)
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
_patch_resolver(monkeypatch, _resolution())
|
||||||
|
|
||||||
|
intent = WEB_MAIN._match_extension_stage_intent("使用 HMM 拓展分析我的文件", "job-x")
|
||||||
|
|
||||||
|
assert intent.get("lightweight") is True
|
||||||
|
assert intent["params"]["extension_id"] == "ext_hmm"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 3. _lightweight_job_file_hints ----------
|
||||||
|
|
||||||
|
def test_job_file_hints_collects_input_and_workspace_relative_paths(
|
||||||
|
monkeypatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
input_root = tmp_path / "input"
|
||||||
|
workspace_root = tmp_path / "workspace"
|
||||||
|
_write(input_root, "job-h1/data/seq.fa", ">a\nAAAA\n")
|
||||||
|
_write(input_root, "job-h1/notes.txt", "notes")
|
||||||
|
_write(workspace_root, "job-h1/out/table.tsv", "a\tb\n")
|
||||||
|
_write(workspace_root, "job-h1/stray.tsv", "x")
|
||||||
|
_write(input_root, "job-other/other.fa", ">o\n") # 其他 job 不应出现
|
||||||
|
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", input_root)
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "WORKSPACE_ROOT", workspace_root)
|
||||||
|
|
||||||
|
hints = WEB_MAIN._lightweight_job_file_hints("job-h1")
|
||||||
|
|
||||||
|
assert "input/job-h1/data/seq.fa" in hints
|
||||||
|
assert "input/job-h1/notes.txt" in hints
|
||||||
|
assert "workspace/job-h1/out/table.tsv" in hints
|
||||||
|
assert "workspace/job-h1/stray.tsv" in hints
|
||||||
|
assert "input/job-other/other.fa" not in hints
|
||||||
|
assert len(hints) == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_job_file_hints_empty_job_id_returns_empty(monkeypatch) -> None:
|
||||||
|
assert WEB_MAIN._lightweight_job_file_hints("") == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_job_file_hints_capped_at_max(monkeypatch, tmp_path: Path) -> None:
|
||||||
|
input_root = tmp_path / "input"
|
||||||
|
for i in range(nl_intent.MAX_JOB_FILE_HINTS + 20):
|
||||||
|
_write(input_root, f"job-cap/f{i:03d}.txt", "x")
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", input_root)
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "WORKSPACE_ROOT", tmp_path / "workspace")
|
||||||
|
|
||||||
|
hints = WEB_MAIN._lightweight_job_file_hints("job-cap")
|
||||||
|
|
||||||
|
assert len(hints) == nl_intent.MAX_JOB_FILE_HINTS
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 4. import-paths 端点 ----------
|
||||||
|
|
||||||
|
def _call_nl_import_paths(job_id: str, paths: list[str]):
|
||||||
|
request = WEB_MAIN.NlImportPathsRequest(job_id=job_id, paths=paths)
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
return asyncio.run(WEB_MAIN.nl_import_paths(request))
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def allowed_source(tmp_path: Path) -> Path:
|
||||||
|
"""在端点允许前缀 /home/xzq/ 下创建临时 source 文件。"""
|
||||||
|
workdir = Path(tempfile.mkdtemp(dir="/home/xzq", prefix="nl-import-test-"))
|
||||||
|
yield workdir
|
||||||
|
shutil.rmtree(workdir, ignore_errors=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_nl_import_paths_copies_file_into_nl_imports(monkeypatch, tmp_path: Path, allowed_source: Path) -> None:
|
||||||
|
source = _write(allowed_source, "family.hmm", "HMM profile content")
|
||||||
|
input_root = tmp_path / "input"
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", input_root)
|
||||||
|
|
||||||
|
result = _call_nl_import_paths("job-imp-1", [str(source)])
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert result["imported"] == {str(source): "input/job-imp-1/_nl_imports/family.hmm"}
|
||||||
|
copied = input_root / "job-imp-1" / "_nl_imports" / "family.hmm"
|
||||||
|
assert copied.is_file()
|
||||||
|
assert copied.read_text(encoding="utf-8") == "HMM profile content"
|
||||||
|
|
||||||
|
|
||||||
|
def test_nl_import_paths_name_conflict_gets_suffix(monkeypatch, tmp_path: Path, allowed_source: Path) -> None:
|
||||||
|
source = _write(allowed_source, "family.hmm", "new content")
|
||||||
|
input_root = tmp_path / "input"
|
||||||
|
existing = _write(input_root, "job-imp-2/_nl_imports/family.hmm", "old content")
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", input_root)
|
||||||
|
|
||||||
|
result = _call_nl_import_paths("job-imp-2", [str(source)])
|
||||||
|
|
||||||
|
assert result["imported"] == {str(source): "input/job-imp-2/_nl_imports/family_1.hmm"}
|
||||||
|
assert existing.read_text(encoding="utf-8") == "old content"
|
||||||
|
copied = input_root / "job-imp-2" / "_nl_imports" / "family_1.hmm"
|
||||||
|
assert copied.is_file()
|
||||||
|
assert copied.read_text(encoding="utf-8") == "new content"
|
||||||
|
|
||||||
|
|
||||||
|
def test_nl_import_paths_rejects_path_outside_allowed_prefixes(
|
||||||
|
monkeypatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", tmp_path / "input")
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
|
with pytest.raises(HTTPException) as exc:
|
||||||
|
_call_nl_import_paths("job-imp-3", ["/etc/passwd"])
|
||||||
|
assert exc.value.status_code == 400
|
||||||
|
assert "路径必须位于" in exc.value.detail
|
||||||
|
|
||||||
|
|
||||||
|
def test_nl_import_paths_rejects_missing_path(monkeypatch, tmp_path: Path) -> None:
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", tmp_path / "input")
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
|
with pytest.raises(HTTPException) as exc:
|
||||||
|
_call_nl_import_paths("job-imp-4", ["/home/xzq/definitely-not-exists-xyz.tsv"])
|
||||||
|
assert exc.value.status_code == 400
|
||||||
|
assert "路径不存在" in exc.value.detail
|
||||||
|
|
||||||
|
|
||||||
|
def test_nl_import_paths_rejects_empty_job_id(monkeypatch, tmp_path: Path) -> None:
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", tmp_path / "input")
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
|
with pytest.raises(HTTPException) as exc:
|
||||||
|
_call_nl_import_paths("", [])
|
||||||
|
assert exc.value.status_code == 400
|
||||||
|
assert "job_id 不能为空" in exc.value.detail
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 5. _run_agent_chat 预闸短路 ----------
|
||||||
|
|
||||||
|
def _call_run_agent_chat(request):
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
return asyncio.run(WEB_MAIN._run_agent_chat(request))
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_agent_chat_short_circuits_on_lightweight_intent(
|
||||||
|
monkeypatch, lightweight_store: Path, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""预闸命中 + 轻量 intent 时 _run_agent_chat 短路返回 propose_extension_stage,
|
||||||
|
且绝不触碰 LLM manager / RouterAgent。"""
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
nl_intent,
|
||||||
|
"resolve_lightweight_extension_intent",
|
||||||
|
lambda message, records, environ=None, client_factory=None, job_file_hints=None: _resolution(),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "llm_manager", None, raising=False)
|
||||||
|
from gfa_core.mas.config import config as mas_config
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mas_config,
|
||||||
|
"load_llm_config",
|
||||||
|
lambda: (_ for _ in ()).throw(RuntimeError("LLM manager must not be touched")),
|
||||||
|
)
|
||||||
|
from gfa_core.mas.agents.router import RouterAgent
|
||||||
|
|
||||||
|
def _boom(_self, _llm_manager, **kwargs):
|
||||||
|
raise RuntimeError("RouterAgent must not be constructed")
|
||||||
|
|
||||||
|
monkeypatch.setattr(RouterAgent, "__init__", _boom)
|
||||||
|
|
||||||
|
result = _call_run_agent_chat(
|
||||||
|
WEB_MAIN.AgentChatRequest(message="使用 HMM 拓展分析我的文件", job_id="job-nl-1")
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert result["status"] == "ok"
|
||||||
|
assert result["action"] == "propose_extension_stage"
|
||||||
|
assert result["intent"].get("lightweight") is True
|
||||||
|
assert result["intent"]["params"]["extension_id"] == "ext_hmm"
|
||||||
|
assert result["model_used"] == "rule"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_agent_chat_does_not_short_circuit_without_lightweight_match(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
"""预闸不命中(消息无触发词)时不查轻量 store、正常走 MainAgent 流程。"""
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
nl_intent,
|
||||||
|
"resolve_lightweight_extension_intent",
|
||||||
|
lambda message, records, environ=None, client_factory=None, job_file_hints=None: nl_intent.LightweightIntentResolution(
|
||||||
|
None, None, {}, {}, None, 0.0, False, "no_trigger", []
|
||||||
|
),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "llm_manager", None, raising=False)
|
||||||
|
from gfa_core.mas.config import config as mas_config
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mas_config,
|
||||||
|
"load_llm_config",
|
||||||
|
lambda: (_ for _ in ()).throw(RuntimeError("LLM manager unavailable (expected path)")),
|
||||||
|
)
|
||||||
|
|
||||||
|
result = _call_run_agent_chat(
|
||||||
|
WEB_MAIN.AgentChatRequest(message="帮我看看这株水稻的基因家族", job_id="job-nl-2")
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "agent_unavailable" # 说明走完了预闸后的 MainAgent 流程
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 6. /api/agent/plan 守卫 ----------
|
||||||
|
|
||||||
|
def _call_agent_plan(request):
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
return asyncio.run(WEB_MAIN.agent_plan(request))
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_plan_guard_skips_extension_response_for_lightweight(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
"""plan 里 extension_intent 带 lightweight=True 时不走 _extension_stage_plan_response。"""
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
_patch_resolver(monkeypatch, _resolution())
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
def _fake_extension_stage_plan_response(extension_intent, session_id, job_id=""):
|
||||||
|
captured["called"] = True
|
||||||
|
return {"_extension_plan_response": True}
|
||||||
|
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "_extension_stage_plan_response", _fake_extension_stage_plan_response)
|
||||||
|
|
||||||
|
result = _call_agent_plan(
|
||||||
|
WEB_MAIN.AgentPlanRequest(
|
||||||
|
message="使用 HMM 拓展分析我的文件",
|
||||||
|
job_id="job-guard",
|
||||||
|
intent={},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "called" not in captured # 守卫拦截,未进入扩展规划响应
|
||||||
|
assert result.get("_extension_plan_response") is not True
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_plan_guard_allows_phase_a_extension_response(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
"""Phase A 命中(非轻量)时 plan 正常进入扩展规划响应。"""
|
||||||
|
monkeypatch.setattr(
|
||||||
|
WEB_MAIN.extension_stage_agent,
|
||||||
|
"list_extensions",
|
||||||
|
lambda include_drafts=False: [_phase_a_ext()],
|
||||||
|
)
|
||||||
|
_write_published_record(lightweight_store, "ext_hmm")
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
def _fake_extension_stage_plan_response(extension_intent, session_id, job_id=""):
|
||||||
|
captured["called"] = True
|
||||||
|
return {"_extension_plan_response": True}
|
||||||
|
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "_extension_stage_plan_response", _fake_extension_stage_plan_response)
|
||||||
|
|
||||||
|
result = _call_agent_plan(
|
||||||
|
WEB_MAIN.AgentPlanRequest(
|
||||||
|
message="使用 hmmsearch 做结构域分析",
|
||||||
|
job_id="job-guard-phase-a",
|
||||||
|
intent={},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert captured.get("called") is True
|
||||||
|
assert result.get("_extension_plan_response") is True
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 8. import-paths 第二轮修复 ----------
|
||||||
|
|
||||||
|
def test_nl_import_paths_directory_name_conflict_gets_suffix(
|
||||||
|
monkeypatch, tmp_path: Path, allowed_source: Path
|
||||||
|
) -> None:
|
||||||
|
source_dir = allowed_source / "mydata"
|
||||||
|
source_dir.mkdir(parents=True)
|
||||||
|
_write(source_dir, "a.txt", "content-a")
|
||||||
|
input_root = tmp_path / "input"
|
||||||
|
existing_dir = input_root / "job-imp-dir" / "_nl_imports" / "mydata"
|
||||||
|
existing_dir.mkdir(parents=True)
|
||||||
|
_write(existing_dir, "old.txt", "old")
|
||||||
|
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", input_root)
|
||||||
|
|
||||||
|
result = _call_nl_import_paths("job-imp-dir", [str(source_dir)])
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert result["imported"] == {str(source_dir): "input/job-imp-dir/_nl_imports/mydata_1"}
|
||||||
|
copied = input_root / "job-imp-dir" / "_nl_imports" / "mydata_1"
|
||||||
|
assert copied.is_dir()
|
||||||
|
assert (copied / "a.txt").read_text(encoding="utf-8") == "content-a"
|
||||||
|
assert (existing_dir / "old.txt").read_text(encoding="utf-8") == "old" # 原目录未被覆盖
|
||||||
|
|
||||||
|
|
||||||
|
def test_nl_import_paths_validates_all_before_copying(
|
||||||
|
monkeypatch, tmp_path: Path, allowed_source: Path
|
||||||
|
) -> None:
|
||||||
|
"""多路径先整体校验后复制:任一非法直接 400,前面的合法路径不被复制。"""
|
||||||
|
good = _write(allowed_source, "good.fa", ">g\nAAAA\n")
|
||||||
|
input_root = tmp_path / "input"
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", input_root)
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
|
with pytest.raises(HTTPException) as exc:
|
||||||
|
_call_nl_import_paths("job-imp-valid", [str(good), "/etc/passwd"])
|
||||||
|
assert exc.value.status_code == 400
|
||||||
|
assert "路径必须位于" in exc.value.detail
|
||||||
|
# 整体校验失败时 good 未被复制
|
||||||
|
assert not (input_root / "job-imp-valid" / "_nl_imports" / "good.fa").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_nl_import_paths_missing_path_validated_before_copying(
|
||||||
|
monkeypatch, tmp_path: Path, allowed_source: Path
|
||||||
|
) -> None:
|
||||||
|
good = _write(allowed_source, "good2.fa", ">g\nAAAA\n")
|
||||||
|
input_root = tmp_path / "input"
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", input_root)
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
|
with pytest.raises(HTTPException) as exc:
|
||||||
|
_call_nl_import_paths("job-imp-valid2", [str(good), "/home/xzq/definitely-not-exists-xyz.tsv"])
|
||||||
|
assert exc.value.status_code == 400
|
||||||
|
assert "路径不存在" in exc.value.detail
|
||||||
|
assert not (input_root / "job-imp-valid2" / "_nl_imports" / "good2.fa").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_nl_import_paths_multiple_valid_paths_all_copied(
|
||||||
|
monkeypatch, tmp_path: Path, allowed_source: Path
|
||||||
|
) -> None:
|
||||||
|
good1 = _write(allowed_source, "one.fa", ">1\n")
|
||||||
|
good2 = _write(allowed_source, "two.fa", ">2\n")
|
||||||
|
input_root = tmp_path / "input"
|
||||||
|
monkeypatch.setattr(WEB_MAIN, "INPUT_ROOT", input_root)
|
||||||
|
|
||||||
|
result = _call_nl_import_paths("job-imp-multi", [str(good1), str(good2)])
|
||||||
|
|
||||||
|
assert result["success"] is True
|
||||||
|
assert set(result["imported"].values()) == {
|
||||||
|
"input/job-imp-multi/_nl_imports/one.fa",
|
||||||
|
"input/job-imp-multi/_nl_imports/two.fa",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_match_lightweight_intent_message_without_summary_hint(
|
||||||
|
monkeypatch, lightweight_store: Path
|
||||||
|
) -> None:
|
||||||
|
"""record 无 summary 时 message 不带括号补充。"""
|
||||||
|
_write_published_record(lightweight_store, "ext_nosum")
|
||||||
|
resolution = nl_intent.LightweightIntentResolution(
|
||||||
|
extension_id="ext_nosum",
|
||||||
|
extension_name="ext_nosum",
|
||||||
|
input_overrides={},
|
||||||
|
parameter_overrides={},
|
||||||
|
explicit_prefix=None,
|
||||||
|
confidence=0.5,
|
||||||
|
prefill=False,
|
||||||
|
reason="ok",
|
||||||
|
candidates_considered=["ext_nosum"],
|
||||||
|
)
|
||||||
|
# 用无 semantic_candidate 的 record 覆盖 fixture
|
||||||
|
record_path = lightweight_store / "extensions" / "ext_nosum" / "extension.json"
|
||||||
|
record = json.loads(record_path.read_text(encoding="utf-8"))
|
||||||
|
record["semantic_candidate"] = {}
|
||||||
|
record_path.write_text(json.dumps(record), encoding="utf-8")
|
||||||
|
_patch_resolver(monkeypatch, resolution)
|
||||||
|
|
||||||
|
intent = WEB_MAIN._match_lightweight_extension_intent("使用拓展分析", "job-x")
|
||||||
|
|
||||||
|
assert intent.get("lightweight") is True
|
||||||
|
assert "已识别为轻量拓展分析:ext_nosum。" in intent["message"]
|
||||||
|
assert "(" not in intent["message"]
|
||||||
@@ -0,0 +1,908 @@
|
|||||||
|
"""轻量版拓展分析自然语言触发 nl_intent 模块单元测试。
|
||||||
|
|
||||||
|
mock 风格对齐 tests/unit/test_extension_stage_lightweight_deepseek_semantics.py:
|
||||||
|
client_factory 注入 + SimpleNamespace 伪造 choices[0].message.content。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from gfa_core.extension_stage_lightweight import nl_intent
|
||||||
|
|
||||||
|
|
||||||
|
def _module():
|
||||||
|
return nl_intent
|
||||||
|
|
||||||
|
|
||||||
|
def _write(root: Path, relative: str, content: str) -> Path:
|
||||||
|
path = root / relative
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(content, encoding="utf-8")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 契约构造 ----------
|
||||||
|
|
||||||
|
def _contract(*, inputs: list[dict] | None = None, parameters: list[dict] | None = None) -> dict:
|
||||||
|
inputs = inputs if inputs is not None else [
|
||||||
|
{
|
||||||
|
"path": "family.hmm",
|
||||||
|
"role": "HMM profile input",
|
||||||
|
"description": "HMM profile file used to search the protein sequences.",
|
||||||
|
"replaceable": True,
|
||||||
|
"locked": False,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"path": "01.col.primary.pep.fa",
|
||||||
|
"role": "Protein sequence input",
|
||||||
|
"description": "Protein FASTA file containing the sequences to search against the HMM profile.",
|
||||||
|
"replaceable": True,
|
||||||
|
"locked": False,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"path": "02_hmm/fixed.tsv",
|
||||||
|
"role": "Locked intermediate",
|
||||||
|
"description": "Intermediate produced inside the run, must not be overridable.",
|
||||||
|
"replaceable": False,
|
||||||
|
"locked": True,
|
||||||
|
},
|
||||||
|
]
|
||||||
|
parameters = parameters if parameters is not None else [
|
||||||
|
{
|
||||||
|
"fact_id": "step_001:parameter:001",
|
||||||
|
"flag": "-p",
|
||||||
|
"display_name": "Output directory",
|
||||||
|
"description": "Directory path to create for HMM search outputs.",
|
||||||
|
"value": "outputs/02_hmm",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"fact_id": "step_002:parameter:003",
|
||||||
|
"flag": "-E",
|
||||||
|
"display_name": "E-value threshold",
|
||||||
|
"description": "E-value threshold for full sequence hits in hmmsearch.",
|
||||||
|
"value": "1e-3",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
return {"inputs": inputs, "parameters": parameters}
|
||||||
|
|
||||||
|
|
||||||
|
def _record(
|
||||||
|
extension_id: str,
|
||||||
|
*,
|
||||||
|
status: str = "published",
|
||||||
|
summary: str = "This tool searches a protein FASTA file against an HMM profile using hmmsearch.",
|
||||||
|
contract: dict | None = None,
|
||||||
|
) -> dict:
|
||||||
|
return {
|
||||||
|
"extension_id": extension_id,
|
||||||
|
"status": status,
|
||||||
|
"semantic_candidate": {
|
||||||
|
"function_summary": summary,
|
||||||
|
},
|
||||||
|
"runtime_binding_contract": contract if contract is not None else _contract(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _good_raw() -> dict:
|
||||||
|
return {
|
||||||
|
"extension_id": "ext_hmm",
|
||||||
|
"input_overrides": [
|
||||||
|
{"path": "family.hmm", "value": "input/job-1/_nl_imports/family.hmm"},
|
||||||
|
{"path": "01.col.primary.pep.fa", "value": "input/job-1/01.col.primary.pep.fa"},
|
||||||
|
],
|
||||||
|
"parameter_overrides": [
|
||||||
|
{"fact_id": "step_002:parameter:003", "value": "1e-5"},
|
||||||
|
],
|
||||||
|
"explicit_prefix": "outputs/nl_run",
|
||||||
|
"confidence": 0.85,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _catalog() -> dict:
|
||||||
|
return _module()._build_catalog([_record("ext_hmm")])
|
||||||
|
|
||||||
|
|
||||||
|
def _response(content: str):
|
||||||
|
return SimpleNamespace(
|
||||||
|
id="request-123",
|
||||||
|
model="deepseek-test",
|
||||||
|
choices=[SimpleNamespace(message=SimpleNamespace(content=content), finish_reason="stop")],
|
||||||
|
usage=SimpleNamespace(prompt_tokens=120, completion_tokens=80, total_tokens=200),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeCompletions:
|
||||||
|
def __init__(self, response=None, error: Exception | None = None) -> None:
|
||||||
|
self.response = response
|
||||||
|
self.error = error
|
||||||
|
self.requests: list[dict] = []
|
||||||
|
|
||||||
|
def create(self, **kwargs):
|
||||||
|
self.requests.append(kwargs)
|
||||||
|
if self.error is not None:
|
||||||
|
raise self.error
|
||||||
|
return self.response
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeClient:
|
||||||
|
def __init__(self, response=None, error: Exception | None = None) -> None:
|
||||||
|
self.completions = _FakeCompletions(response=response, error=error)
|
||||||
|
self.chat = SimpleNamespace(completions=self.completions)
|
||||||
|
|
||||||
|
|
||||||
|
class _CountingFactory:
|
||||||
|
def __init__(self, client) -> None:
|
||||||
|
self.client = client
|
||||||
|
self.calls = 0
|
||||||
|
self.configs: list[Any] = []
|
||||||
|
|
||||||
|
def __call__(self, config):
|
||||||
|
self.calls += 1
|
||||||
|
self.configs.append(config)
|
||||||
|
return self.client
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 1. 粗筛不中不调 LLM ----------
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"message",
|
||||||
|
[
|
||||||
|
"帮我看看这株水稻的基因家族有哪些",
|
||||||
|
"请绘制进化树并做共线性分析",
|
||||||
|
"这个蛋白序列应该怎么注释",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_no_trigger_message_resolves_none_and_never_calls_client(message: str) -> None:
|
||||||
|
records = [_record("ext_hmm")]
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(_good_raw())))
|
||||||
|
factory = _CountingFactory(fake)
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
message, records, environ={}, client_factory=factory
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id is None
|
||||||
|
assert resolution.reason == "no_trigger"
|
||||||
|
assert factory.calls == 0
|
||||||
|
assert fake.completions.requests == []
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 2. 触发词命中但无 records ----------
|
||||||
|
|
||||||
|
def test_trigger_without_records_never_calls_llm() -> None:
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(_good_raw())))
|
||||||
|
factory = _CountingFactory(fake)
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"使用拓展分析处理我的文件", [], environ={}, client_factory=factory
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id is None
|
||||||
|
assert resolution.reason == "no_published_records"
|
||||||
|
assert factory.calls == 0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 3. _build_catalog ----------
|
||||||
|
|
||||||
|
def test_build_catalog_only_replaceable_inputs_without_locked_and_parameter_values() -> None:
|
||||||
|
catalog = _module()._build_catalog([_record("ext_hmm")])
|
||||||
|
|
||||||
|
assert catalog["truncated"] is False
|
||||||
|
entry = catalog["extensions"][0]
|
||||||
|
assert entry["extension_id"] == "ext_hmm"
|
||||||
|
assert entry["function_summary"].startswith("This tool searches a protein FASTA")
|
||||||
|
|
||||||
|
input_keys = {tuple(sorted(item.keys())) for item in entry["inputs"]}
|
||||||
|
assert input_keys == {("description", "path", "role")}
|
||||||
|
paths = {item["path"] for item in entry["inputs"]}
|
||||||
|
assert "family.hmm" in paths
|
||||||
|
assert "01.col.primary.pep.fa" in paths
|
||||||
|
assert "02_hmm/fixed.tsv" not in paths # locked 输入被剔除
|
||||||
|
|
||||||
|
parameter_keys = {tuple(sorted(item.keys())) for item in entry["parameters"]}
|
||||||
|
assert parameter_keys == {("description", "display_name", "fact_id", "flag")}
|
||||||
|
assert all("value" not in item for item in entry["parameters"])
|
||||||
|
flags = {item["flag"] for item in entry["parameters"]}
|
||||||
|
assert flags == {"-p", "-E"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_catalog_truncates_when_over_max_chars() -> None:
|
||||||
|
module = _module()
|
||||||
|
long_summary = "x" * 500
|
||||||
|
record = _record("ext_long", summary=long_summary)
|
||||||
|
single_size = len(json.dumps(module._build_catalog([record])["extensions"][0], ensure_ascii=False))
|
||||||
|
|
||||||
|
assert single_size < module.MAX_CATALOG_CHARS # 单条不足以触发截断
|
||||||
|
|
||||||
|
records = [_record(f"ext_{i:02d}", summary=long_summary) for i in range(60)]
|
||||||
|
catalog = module._build_catalog(records)
|
||||||
|
|
||||||
|
assert catalog["truncated"] is True
|
||||||
|
included = len(catalog["extensions"])
|
||||||
|
assert included < len(records)
|
||||||
|
assert included >= 1
|
||||||
|
total = sum(len(json.dumps(item, ensure_ascii=False)) for item in catalog["extensions"])
|
||||||
|
assert total <= module.MAX_CATALOG_CHARS
|
||||||
|
# 截断点之后的首条记录(若追加)必然超限
|
||||||
|
first_excluded = records[included]["extension_id"]
|
||||||
|
extra = len(json.dumps(module._build_catalog([records[included]])["extensions"][0], ensure_ascii=False))
|
||||||
|
assert total + extra > module.MAX_CATALOG_CHARS
|
||||||
|
assert first_excluded not in [item["extension_id"] for item in catalog["extensions"]]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 4. _sanitize_resolution ----------
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("mutate", "fragment"),
|
||||||
|
[
|
||||||
|
(lambda raw: raw.update({"input_overrides": [{"path": "invented.fa", "value": "input/job-1/invented.fa"}]}),
|
||||||
|
"input_overrides 引用了不存在的 path"),
|
||||||
|
(lambda raw: raw.update({"parameter_overrides": [{"fact_id": "step_099:parameter:099", "value": "x"}]}),
|
||||||
|
"parameter_overrides 引用了不存在的 fact_id"),
|
||||||
|
(lambda raw: raw.update({"extension_id": "ext_invented"}), "extension_id 不在可选目录中"),
|
||||||
|
(lambda raw: raw.update({"parameter_overrides": [{"fact_id": "step_002:parameter:003", "value": 7}]}),
|
||||||
|
"的 value 必须是非空字符串"),
|
||||||
|
(lambda raw: raw.update({"confidence": "high"}), "confidence 不是数字"),
|
||||||
|
(lambda raw: raw.update({"input_overrides": "family.hmm"}), "input_overrides 必须是数组"),
|
||||||
|
(lambda raw: raw.update({"input_overrides": [{"path": "family.hmm", "value": ""}]}),
|
||||||
|
"的 value 必须是非空字符串"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_sanitize_rejects_invalid_resolution(mutate, fragment: str) -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
mutate(raw)
|
||||||
|
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
|
||||||
|
assert sanitized is None
|
||||||
|
assert reason is not None
|
||||||
|
assert fragment in reason
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"raw",
|
||||||
|
[
|
||||||
|
"not a json object",
|
||||||
|
["extension_id"],
|
||||||
|
None,
|
||||||
|
42,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_sanitize_rejects_non_object_payloads(raw) -> None:
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
assert sanitized is None
|
||||||
|
assert reason is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_sanitize_accepts_valid_resolution_and_maps_fields() -> None:
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(_good_raw(), _catalog())
|
||||||
|
|
||||||
|
assert reason is None
|
||||||
|
assert sanitized is not None
|
||||||
|
assert sanitized["extension_id"] == "ext_hmm"
|
||||||
|
assert sanitized["input_overrides"] == {
|
||||||
|
"family.hmm": "input/job-1/_nl_imports/family.hmm",
|
||||||
|
"01.col.primary.pep.fa": "input/job-1/01.col.primary.pep.fa",
|
||||||
|
}
|
||||||
|
assert sanitized["parameter_overrides"] == {"step_002:parameter:003": "1e-5"}
|
||||||
|
assert sanitized["explicit_prefix"] == "outputs/nl_run"
|
||||||
|
assert sanitized["confidence"] == 0.85
|
||||||
|
|
||||||
|
|
||||||
|
def test_sanitize_accepts_parameter_override_even_when_user_message_omits_it() -> None:
|
||||||
|
"""sanitize 只做结构校验;'参数值必须来自用户消息' 的约束在 prompt 层,不做程序级校验。"""
|
||||||
|
raw = _good_raw() # 参数 1e-5 未在用户消息中出现(消息由调用方构造)
|
||||||
|
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
|
||||||
|
assert reason is None
|
||||||
|
assert sanitized is not None
|
||||||
|
assert sanitized["parameter_overrides"] == {"step_002:parameter:003": "1e-5"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_sanitize_allows_missing_optional_fields() -> None:
|
||||||
|
raw = {"extension_id": "ext_hmm", "confidence": None}
|
||||||
|
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
|
||||||
|
assert reason is None
|
||||||
|
assert sanitized is not None
|
||||||
|
assert sanitized["input_overrides"] == {}
|
||||||
|
assert sanitized["parameter_overrides"] == {}
|
||||||
|
assert sanitized["explicit_prefix"] is None
|
||||||
|
assert sanitized["confidence"] == 0.0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 5. LLM 异常 / 坏 JSON / 缺 choices ----------
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("content", "error"),
|
||||||
|
[
|
||||||
|
("this is not json", None), # 坏 JSON
|
||||||
|
("{\"hello\": \"world\"}", None), # 合法 JSON 但字段缺失
|
||||||
|
(None, RuntimeError("connection reset")), # LLM 抛异常
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_llm_failures_degrade_to_none_resolution(content, error) -> None:
|
||||||
|
fake = _FakeClient(response=_response(content) if content is not None else None, error=error)
|
||||||
|
factory = _CountingFactory(fake)
|
||||||
|
records = [_record("ext_hmm")]
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"使用 HMM 拓展分析对刚上传的文件进行分析",
|
||||||
|
records,
|
||||||
|
environ={},
|
||||||
|
client_factory=factory,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id is None
|
||||||
|
assert resolution.reason == "llm_unavailable_or_invalid" or resolution.reason.startswith("invalid_resolution:")
|
||||||
|
assert factory.calls == 1
|
||||||
|
|
||||||
|
|
||||||
|
class _NoChoicesClient:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.chat = SimpleNamespace(
|
||||||
|
completions=SimpleNamespace(
|
||||||
|
create=lambda **kwargs: SimpleNamespace(choices=[]),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_choices_degrades_to_none_resolution() -> None:
|
||||||
|
factory = _CountingFactory(_NoChoicesClient())
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"使用 HMM 拓展分析对刚上传的文件进行分析",
|
||||||
|
[_record("ext_hmm")],
|
||||||
|
environ={},
|
||||||
|
client_factory=factory,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id is None
|
||||||
|
assert resolution.reason == "llm_unavailable_or_invalid"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 6. happy path ----------
|
||||||
|
|
||||||
|
def test_happy_path_resolves_all_fields_and_prefills() -> None:
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(_good_raw())))
|
||||||
|
factory = _CountingFactory(fake)
|
||||||
|
records = [_record("ext_hmm")]
|
||||||
|
message = "使用 HMM 拓展分析对刚上传的文件进行分析,参数 e-value=1e-5"
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
message, records, environ={}, client_factory=factory
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id == "ext_hmm"
|
||||||
|
assert resolution.extension_name == "ext_hmm"
|
||||||
|
assert resolution.input_overrides == {
|
||||||
|
"family.hmm": "input/job-1/_nl_imports/family.hmm",
|
||||||
|
"01.col.primary.pep.fa": "input/job-1/01.col.primary.pep.fa",
|
||||||
|
}
|
||||||
|
assert resolution.parameter_overrides == {"step_002:parameter:003": "1e-5"}
|
||||||
|
assert resolution.explicit_prefix == "outputs/nl_run"
|
||||||
|
assert resolution.confidence == 0.85
|
||||||
|
assert resolution.prefill is True
|
||||||
|
assert resolution.reason == "ok"
|
||||||
|
assert resolution.candidates_considered == ["ext_hmm"]
|
||||||
|
assert factory.calls == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_high_confidence_without_overrides_does_not_prefill() -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["input_overrides"] = []
|
||||||
|
raw["parameter_overrides"] = []
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(raw)))
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"使用 HMM 拓展分析", [_record("ext_hmm")], environ={}, client_factory=lambda config: fake
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id == "ext_hmm"
|
||||||
|
assert resolution.confidence >= _module().PREFILL_THRESHOLD
|
||||||
|
assert resolution.prefill is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_low_confidence_does_not_prefill() -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["confidence"] = 0.3
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(raw)))
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"使用 HMM 拓展分析", [_record("ext_hmm")], environ={}, client_factory=lambda config: fake
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id == "ext_hmm"
|
||||||
|
assert resolution.prefill is False
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 7. load_published_lightweight_records ----------
|
||||||
|
|
||||||
|
def test_load_published_records_only_returns_published(monkeypatch, tmp_path: Path) -> None:
|
||||||
|
_write(tmp_path, "extensions/ext_pub/extension.json", json.dumps(_record("ext_pub", status="published")))
|
||||||
|
_write(tmp_path, "extensions/ext_pending/extension.json", json.dumps(_record("ext_pending", status="confirmation_pending")))
|
||||||
|
_write(tmp_path, "extensions/ext_failed/extension.json", json.dumps(_record("ext_failed", status="baseline_failed")))
|
||||||
|
_write(tmp_path, "extensions/ext_bad/extension.json", "{bad json")
|
||||||
|
_write(tmp_path, "extensions/ext_notdict/extension.json", "[1, 2, 3]")
|
||||||
|
_write(tmp_path, "extensions/ext_other/list.txt", "{}")
|
||||||
|
|
||||||
|
records = _module().load_published_lightweight_records(tmp_path)
|
||||||
|
|
||||||
|
ids = [str(r["extension_id"]) for r in records]
|
||||||
|
assert ids == ["ext_pub"]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 8. job_file_hints 进入 LLM user 消息 ----------
|
||||||
|
|
||||||
|
def test_job_file_hints_reach_llm_user_message() -> None:
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(_good_raw())))
|
||||||
|
factory = _CountingFactory(fake)
|
||||||
|
|
||||||
|
_module().resolve_lightweight_extension_intent(
|
||||||
|
"使用 HMM 拓展分析对刚上传的文件进行分析",
|
||||||
|
[_record("ext_hmm")],
|
||||||
|
environ={},
|
||||||
|
client_factory=factory,
|
||||||
|
job_file_hints=["input/job-1/01.col.primary.pep.fa", "workspace/job-1/out/table.tsv"],
|
||||||
|
)
|
||||||
|
|
||||||
|
request = fake.completions.requests[0]
|
||||||
|
user_message = next(item["content"] for item in request["messages"] if item["role"] == "user")
|
||||||
|
payload = json.loads(user_message)
|
||||||
|
assert payload["user_message"] == "使用 HMM 拓展分析对刚上传的文件进行分析"
|
||||||
|
assert payload["job_files"] == ["input/job-1/01.col.primary.pep.fa", "workspace/job-1/out/table.tsv"]
|
||||||
|
assert payload["extensions"][0]["extension_id"] == "ext_hmm"
|
||||||
|
|
||||||
|
|
||||||
|
def test_job_file_hints_are_capped() -> None:
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(_good_raw())))
|
||||||
|
|
||||||
|
_module().resolve_lightweight_extension_intent(
|
||||||
|
"使用 HMM 拓展分析",
|
||||||
|
[_record("ext_hmm")],
|
||||||
|
environ={},
|
||||||
|
client_factory=lambda config: fake,
|
||||||
|
job_file_hints=[f"input/job-1/file_{i}.txt" for i in range(500)],
|
||||||
|
)
|
||||||
|
|
||||||
|
request = fake.completions.requests[0]
|
||||||
|
user_message = next(item["content"] for item in request["messages"] if item["role"] == "user")
|
||||||
|
payload = json.loads(user_message)
|
||||||
|
assert len(payload["job_files"]) == _module().MAX_JOB_FILE_HINTS
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 9. 粗筛行为 ----------
|
||||||
|
|
||||||
|
def test_rough_candidates_without_records_is_empty() -> None:
|
||||||
|
assert _module()._rough_candidates("拓展分析我的文件", []) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_trigger_without_records_is_false() -> None:
|
||||||
|
assert _module()._rough_trigger("拓展分析我的文件", []) is False
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("message", "expected_ids"),
|
||||||
|
[
|
||||||
|
# 强触发词且无关键词命中:全量兜底
|
||||||
|
("帮我做拓展分析", ["ext_hmm", "ext_dia"]),
|
||||||
|
("帮我做扩展分析", ["ext_hmm", "ext_dia"]),
|
||||||
|
("do an extension analysis", ["ext_hmm", "ext_dia"]),
|
||||||
|
# 工具词命中子集(不再全量)
|
||||||
|
("用 hmmsearch 搜索结构域", ["ext_hmm"]),
|
||||||
|
# 工具词无关键词命中:空(不再全量);用不含 blast/hmmer 相关词的记录验证
|
||||||
|
("run blast against the library", []),
|
||||||
|
("run hmmer on my file", []),
|
||||||
|
# 英文关键词命中单个
|
||||||
|
("diamond align my sequences", ["ext_dia"]),
|
||||||
|
("evalue cutoff please", ["ext_dia"]),
|
||||||
|
# 无关消息
|
||||||
|
("帮我看看这株水稻的基因家族", []),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_rough_candidates(message: str, expected_ids: list[str]) -> None:
|
||||||
|
if message.startswith("run "):
|
||||||
|
# 工具词无关键词命中:用不含相关词的记录,确保结果为 []
|
||||||
|
records = [
|
||||||
|
_record("x9_zz", summary="Sorts entries in parallel for a table."),
|
||||||
|
_record("x9_yy", summary="Counts rows in a matrix."),
|
||||||
|
]
|
||||||
|
assert _module()._rough_candidates(message, records) == expected_ids
|
||||||
|
return
|
||||||
|
records = [
|
||||||
|
_record("ext_hmm", summary="This tool searches a protein FASTA file against an HMM profile using hmmsearch."),
|
||||||
|
_record("ext_dia", summary="This tool aligns sequences with diamond using an e-value cutoff."),
|
||||||
|
]
|
||||||
|
assert _module()._rough_candidates(message, records) == expected_ids
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_candidates_reject_summary_stopwords_only() -> None:
|
||||||
|
"""summary 只含停用词时,无关消息不应命中该记录。"""
|
||||||
|
records = [_record("ext_generic", summary="This tool analyzes sequence files using a data tool.")]
|
||||||
|
assert _module()._rough_candidates("帮我看看这株水稻的基因家族", records) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_candidates_chinese_summary_substrings() -> None:
|
||||||
|
record = _record("ext_zh", summary="使用样本表校验工具对测序数据进行质控。")
|
||||||
|
assert "ext_zh" in _module()._rough_candidates("帮我做样本表校验", [record])
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 10. resolve 对空 catalog 的容错 ----------
|
||||||
|
|
||||||
|
def test_resolve_degrades_when_catalog_empty_after_rough_match() -> None:
|
||||||
|
record = _record("ext_broken")
|
||||||
|
record["runtime_binding_contract"] = None # 粗筛命中(有 summary 关键词)但目录构造为空
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(_good_raw())))
|
||||||
|
factory = _CountingFactory(fake)
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"hmmsearch tool analysis", [record], environ={}, client_factory=factory
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id is None
|
||||||
|
assert factory.calls == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolution_uses_compact_extension_name() -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["extension_id"] = "03___hmm_1787130993548"
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(raw)))
|
||||||
|
summary = "Searches profiles. It then parses the domain table and reports hits."
|
||||||
|
record = _record("03___hmm_1787130993548", summary=summary)
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"使用拓展分析", [record], environ={}, client_factory=lambda config: fake
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_name == "03___hmm"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 11. 真实 store 冒烟 ----------
|
||||||
|
|
||||||
|
def test_real_store_catalog_truncation_smoke() -> None:
|
||||||
|
"""用远端真实 store 验证目录截断现象(数据源随仓库存在)。"""
|
||||||
|
store_root = Path(__file__).resolve().parents[2] / ".runtime/lightweight-extension/store"
|
||||||
|
records = _module().load_published_lightweight_records(store_root)
|
||||||
|
if not records:
|
||||||
|
pytest.skip("真实 store 不可用(无 published 记录)")
|
||||||
|
catalog = _module()._build_catalog(records)
|
||||||
|
all_ids = sorted(str(r.get("extension_id")) for r in records)
|
||||||
|
included_ids = [item["extension_id"] for item in catalog["extensions"]]
|
||||||
|
assert catalog["truncated"] is True
|
||||||
|
assert set(included_ids) < set(all_ids)
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_trigger_accepts_real_published_records() -> None:
|
||||||
|
store_root = Path(__file__).resolve().parents[2] / ".runtime/lightweight-extension/store"
|
||||||
|
records = _module().load_published_lightweight_records(store_root)
|
||||||
|
if not records:
|
||||||
|
pytest.skip("真实 store 不可用(无 published 记录)")
|
||||||
|
assert _module()._rough_trigger("帮我做拓展分析", records) is True
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 12. 分支补测(覆盖率补充) ----------
|
||||||
|
|
||||||
|
def test_load_published_records_bad_store_root_returns_empty() -> None:
|
||||||
|
# Path 构造带 NUL 字节会抛 ValueError,走 except 分支
|
||||||
|
assert _module().load_published_lightweight_records(5) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_function_summary_text_missing_semantic_candidate_returns_empty() -> None:
|
||||||
|
record = _record("ext_none")
|
||||||
|
record["semantic_candidate"] = None
|
||||||
|
assert _module()._function_summary_text(record) == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_catalog_skips_record_without_extension_id() -> None:
|
||||||
|
record = _record("")
|
||||||
|
catalog = _module()._build_catalog([record])
|
||||||
|
assert catalog["extensions"] == []
|
||||||
|
assert catalog["truncated"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_sanitize_rejects_non_mapping_input_override_element() -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["input_overrides"] = [42]
|
||||||
|
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
|
||||||
|
assert sanitized is None
|
||||||
|
assert reason == "input_overrides 元素必须是对象"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sanitize_rejects_parameter_overrides_not_list() -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["parameter_overrides"] = {"step_002:parameter:003": "1e-5"}
|
||||||
|
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
|
||||||
|
assert sanitized is None
|
||||||
|
assert reason == "parameter_overrides 必须是数组"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sanitize_rejects_non_mapping_parameter_element() -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["parameter_overrides"] = ["1e-5"]
|
||||||
|
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
|
||||||
|
assert sanitized is None
|
||||||
|
assert reason == "parameter_overrides 元素必须是对象"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sanitize_rejects_non_string_explicit_prefix() -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["explicit_prefix"] = 42
|
||||||
|
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
|
||||||
|
assert sanitized is None
|
||||||
|
assert reason == "explicit_prefix 必须是字符串或 null"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sanitize_rejects_confidence_out_of_range() -> None:
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["confidence"] = 1.5
|
||||||
|
|
||||||
|
sanitized, reason = _module()._sanitize_resolution(raw, _catalog())
|
||||||
|
|
||||||
|
assert sanitized is None
|
||||||
|
assert reason == "confidence 超出 [0,1]"
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_client_path_uses_configured_env_file(monkeypatch, tmp_path: Path) -> None:
|
||||||
|
"""environ 提供有效 env 文件且未注入 client_factory 时走 _default_client 分支。"""
|
||||||
|
from gfa_core.extension_stage_lightweight import deepseek_semantics
|
||||||
|
|
||||||
|
env_file = _write(
|
||||||
|
tmp_path,
|
||||||
|
".env",
|
||||||
|
"NEW_API_BASE_URL=https://api.example.test/v1\nNEW_API_MODEL=deepseek-test\nNEW_API_KEY=secret-test-key\n",
|
||||||
|
)
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(_good_raw())))
|
||||||
|
monkeypatch.setattr(deepseek_semantics, "_default_client", lambda config: fake)
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"使用 HMM 拓展分析",
|
||||||
|
[_record("ext_hmm")],
|
||||||
|
environ={"BBAV5_PROJECT_ENV_FILE": str(env_file)},
|
||||||
|
client_factory=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id == "ext_hmm"
|
||||||
|
assert fake.completions.requests
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_client_without_config_degrades(monkeypatch) -> None:
|
||||||
|
"""environ 无有效配置且未注入 client_factory 时,默认客户端分支因 config=None 降级。"""
|
||||||
|
from gfa_core.extension_stage_lightweight import deepseek_semantics
|
||||||
|
|
||||||
|
monkeypatch.setattr(deepseek_semantics, "load_deepseek_config", lambda environ: None)
|
||||||
|
|
||||||
|
resolution = _module().resolve_lightweight_extension_intent(
|
||||||
|
"使用 HMM 拓展分析",
|
||||||
|
[_record("ext_hmm")],
|
||||||
|
environ={},
|
||||||
|
client_factory=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert resolution.extension_id is None
|
||||||
|
assert resolution.reason == "llm_unavailable_or_invalid"
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_store_pure_chinese_function_words_miss_rough_candidates() -> None:
|
||||||
|
"""R2 实测固化:function_summary 全英文,纯中文功能词(无强触发词)不命中粗筛、不调 LLM。"""
|
||||||
|
store_root = Path(__file__).resolve().parents[2] / ".runtime/lightweight-extension/store"
|
||||||
|
records = _module().load_published_lightweight_records(store_root)
|
||||||
|
if not records:
|
||||||
|
pytest.skip("真实 store 不可用(无 published 记录)")
|
||||||
|
|
||||||
|
assert _module()._rough_candidates("帮我做样本表校验", records) == []
|
||||||
|
assert _module()._rough_trigger("帮我做样本表校验", records) is False
|
||||||
|
# 强触发词下全部记录进候选(61 个),目录截断导致 LLM 看不到后段记录
|
||||||
|
assert len(_module()._rough_candidates("用拓展分析做分析", records)) == len(records)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 13. 第二轮:union 粗筛 / compact 名称 / catalog 24000 ----------
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("extension_id", "expected"),
|
||||||
|
[
|
||||||
|
("03___hmm_1787130993548", "03___hmm"),
|
||||||
|
("10_pangene__pangene_1786213304494", "10_pangene__pangene"),
|
||||||
|
("ext_no_timestamp", "ext_no_timestamp"),
|
||||||
|
("", ""),
|
||||||
|
("x" * 40, "x" * 30),
|
||||||
|
("12_1787211341321", "12"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_compact_extension_name(extension_id: str, expected: str) -> None:
|
||||||
|
assert _module().compact_extension_name(extension_id) == expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_extension_id_tokens() -> None:
|
||||||
|
record = _record("03___hmm_1787130993548")
|
||||||
|
tokens = _module()._extension_id_tokens(record)
|
||||||
|
assert "hmm" in tokens
|
||||||
|
assert "03" not in tokens # len<3 剔除
|
||||||
|
|
||||||
|
|
||||||
|
def test_parameter_flag_tokens() -> None:
|
||||||
|
record = _record("ext_hmm")
|
||||||
|
record["runtime_binding_contract"]["parameters"] = [
|
||||||
|
{"flag": "--samples", "fact_id": "p1"},
|
||||||
|
{"flag": "-e", "fact_id": "p2"}, # len<3 剔除
|
||||||
|
{"flag": "--E", "fact_id": "p3"}, # 单字母剔除
|
||||||
|
{"flag": "output", "fact_id": "p4"}, # 英文停用词剔除
|
||||||
|
]
|
||||||
|
tokens = _module()._parameter_flag_tokens(record)
|
||||||
|
assert "samples" in tokens
|
||||||
|
assert "e" not in tokens
|
||||||
|
assert "output" not in tokens
|
||||||
|
|
||||||
|
|
||||||
|
def test_parameter_flag_tokens_no_contract() -> None:
|
||||||
|
record = _record("ext_hmm")
|
||||||
|
record["runtime_binding_contract"] = None
|
||||||
|
assert _module()._parameter_flag_tokens(record) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_record_keyword_sources_merges_all_sources() -> None:
|
||||||
|
record = _record("03___hmm_1787130993548", summary="Aligns with diamond using e-value.")
|
||||||
|
record["runtime_binding_contract"]["parameters"] = [{"flag": "--samples"}]
|
||||||
|
sources = _module()._record_keyword_sources(record)
|
||||||
|
assert "hmm" in sources # extension_id token
|
||||||
|
assert "diamond" in sources # summary token
|
||||||
|
assert "samples" in sources # parameter flag
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_candidates_union_strategy_with_strong_trigger_and_keyword() -> None:
|
||||||
|
"""带工具词 + 强触发词 → 关键词子集(不因强触发词全量)。"""
|
||||||
|
records = [
|
||||||
|
_record("ext_hmm", summary="This tool searches a protein FASTA file against an HMM profile using hmmsearch."),
|
||||||
|
_record("ext_dia", summary="This tool aligns sequences with diamond using an e-value cutoff."),
|
||||||
|
]
|
||||||
|
cands = _module()._rough_candidates("用 hmmsearch 拓展分析我的文件", records)
|
||||||
|
assert cands == ["ext_hmm"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_candidates_pure_strong_trigger_falls_back_to_all() -> None:
|
||||||
|
records = [
|
||||||
|
_record("ext_hmm", summary="This tool searches a protein FASTA file against an HMM profile using hmmsearch."),
|
||||||
|
_record("ext_dia", summary="This tool aligns sequences with diamond using an e-value cutoff."),
|
||||||
|
]
|
||||||
|
cands = _module()._rough_candidates("帮我做拓展分析", records)
|
||||||
|
assert sorted(cands) == ["ext_dia", "ext_hmm"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_candidates_no_match_returns_empty() -> None:
|
||||||
|
records = [_record("ext_hmm", summary="This tool searches a protein FASTA file against an HMM profile using hmmsearch.")]
|
||||||
|
assert _module()._rough_candidates("帮我看看这株水稻的基因家族", records) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_candidates_skips_missing_extension_id() -> None:
|
||||||
|
"""缺失 extension_id 的记录被过滤,不产出 None 候选。"""
|
||||||
|
no_id = _record("")
|
||||||
|
no_id["extension_id"] = None
|
||||||
|
records = [
|
||||||
|
no_id,
|
||||||
|
_record("ext_hmm", summary="This tool searches a protein FASTA file against an HMM profile using hmmsearch."),
|
||||||
|
]
|
||||||
|
cands = _module()._rough_candidates("hmmsearch 结构域", records)
|
||||||
|
assert cands == ["ext_hmm"]
|
||||||
|
assert "None" not in cands
|
||||||
|
assert None not in cands
|
||||||
|
|
||||||
|
|
||||||
|
def test_rough_candidates_sorted_by_hits_desc() -> None:
|
||||||
|
"""命中关键词数降序(同分按 id 排序)。"""
|
||||||
|
records = [
|
||||||
|
_record("ext_a", summary="diamond e-value cutoff search"),
|
||||||
|
_record("ext_b", summary="diamond align"),
|
||||||
|
]
|
||||||
|
cands = _module()._rough_candidates("diamond e-value", records)
|
||||||
|
assert cands == ["ext_a", "ext_b"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_catalog_uses_max_24000() -> None:
|
||||||
|
module = _module()
|
||||||
|
assert module.MAX_CATALOG_CHARS == 24000
|
||||||
|
long_summary = "x" * 500
|
||||||
|
records = [_record(f"ext_{i:02d}", summary=long_summary) for i in range(60)]
|
||||||
|
catalog = module._build_catalog(records)
|
||||||
|
assert catalog["truncated"] is True
|
||||||
|
included = len(catalog["extensions"])
|
||||||
|
assert included < len(records)
|
||||||
|
total = sum(len(json.dumps(item, ensure_ascii=False)) for item in catalog["extensions"])
|
||||||
|
assert total <= module.MAX_CATALOG_CHARS
|
||||||
|
first_excluded = records[included]["extension_id"]
|
||||||
|
extra = len(json.dumps(module._build_catalog([records[included]])["extensions"][0], ensure_ascii=False))
|
||||||
|
assert total + extra > module.MAX_CATALOG_CHARS
|
||||||
|
assert first_excluded not in [item["extension_id"] for item in catalog["extensions"]]
|
||||||
|
|
||||||
|
|
||||||
|
def test_catalog_dedupes_duplicate_ids() -> None:
|
||||||
|
module = _module()
|
||||||
|
records = [_record("ext_dup"), _record("ext_dup")]
|
||||||
|
catalog = module._build_catalog(records)
|
||||||
|
assert len(catalog["extensions"]) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_truncated_catalog_sets_reason_hint() -> None:
|
||||||
|
"""目录截断时 resolution.reason 注明请补充工具名。"""
|
||||||
|
module = _module()
|
||||||
|
records = [_record(f"ext_{i:02d}", summary="x" * 500) for i in range(60)]
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["extension_id"] = "ext_00" # catalog 首个 id(命中数降序后)
|
||||||
|
# 纯强触发词全量兜底 → catalog 截断 → reason 带提示
|
||||||
|
resolution = module.resolve_lightweight_extension_intent(
|
||||||
|
"帮我做拓展分析",
|
||||||
|
records,
|
||||||
|
environ={},
|
||||||
|
client_factory=lambda config: _FakeClient(response=_response(json.dumps(raw))),
|
||||||
|
)
|
||||||
|
assert resolution.reason.startswith("ok; 候选过多已截断")
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_orders_records_by_candidate_score() -> None:
|
||||||
|
"""catalog 顺序跟随 _rough_candidates 的命中数降序。"""
|
||||||
|
module = _module()
|
||||||
|
records = [
|
||||||
|
_record("ext_a", summary="diamond e-value cutoff search"),
|
||||||
|
_record("ext_b", summary="diamond align"),
|
||||||
|
]
|
||||||
|
raw = _good_raw()
|
||||||
|
raw["extension_id"] = "ext_a"
|
||||||
|
fake = _FakeClient(response=_response(json.dumps(raw)))
|
||||||
|
|
||||||
|
resolution = module.resolve_lightweight_extension_intent(
|
||||||
|
"diamond e-value", records, environ={}, client_factory=lambda config: fake
|
||||||
|
)
|
||||||
|
assert resolution.extension_id == "ext_a"
|
||||||
|
request = fake.completions.requests[0]
|
||||||
|
user_message = next(item["content"] for item in request["messages"] if item["role"] == "user")
|
||||||
|
payload = json.loads(user_message)
|
||||||
|
ids = [e["extension_id"] for e in payload["extensions"]]
|
||||||
|
assert ids[0] == "ext_a"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parameter_flag_tokens_skips_non_mapping_items() -> None:
|
||||||
|
"""parameters 列表含非 dict 元素时跳过。"""
|
||||||
|
record = _record("ext_hmm")
|
||||||
|
record["runtime_binding_contract"]["parameters"] = [
|
||||||
|
{"flag": "--samples", "fact_id": "p1"},
|
||||||
|
"not-a-dict",
|
||||||
|
42,
|
||||||
|
None,
|
||||||
|
]
|
||||||
|
tokens = _module()._parameter_flag_tokens(record)
|
||||||
|
assert tokens == ["samples"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_store_pm_alignment_counts() -> None:
|
||||||
|
"""PM 实测对齐固化:HMM→9、DIAMOND→8、OrthoFinder→3、KaKs→2、ManifestQC→5、纯拓展→61。"""
|
||||||
|
store_root = Path(__file__).resolve().parents[2] / ".runtime/lightweight-extension/store"
|
||||||
|
records = _module().load_published_lightweight_records(store_root)
|
||||||
|
if len(records) != 61:
|
||||||
|
pytest.skip(f"真实 store 记录数变化(当前 {len(records)},基线 61),跳过 PM 对齐断言")
|
||||||
|
|
||||||
|
assert len(_module()._rough_candidates("帮我用 HMM 拓展分析我的文件", records)) == 9
|
||||||
|
assert len(_module()._rough_candidates("帮我用 DIAMOND 拓展分析", records)) == 8
|
||||||
|
assert len(_module()._rough_candidates("用 OrthoFinder 拓展分析", records)) == 3
|
||||||
|
assert len(_module()._rough_candidates("用 KaKs 拓展分析", records)) == 2
|
||||||
|
assert len(_module()._rough_candidates("用 ManifestQC 拓展分析", records)) == 5
|
||||||
|
assert len(_module()._rough_candidates("帮我做拓展分析", records)) == 61
|
||||||
|
# HMM 子集全为 03_ 系列(hmm 关键词),且截断在子集内不发生
|
||||||
|
hmm = _module()._rough_candidates("帮我用 HMM 拓展分析我的文件", records)
|
||||||
|
assert all(c.startswith("03___hmm") for c in hmm[:4])
|
||||||
|
catalog = _module()._build_catalog(
|
||||||
|
[r for r in records if str(r.get("extension_id")) in hmm]
|
||||||
|
)
|
||||||
|
assert catalog["truncated"] is False
|
||||||
Reference in New Issue
Block a user