PlantAI-dev/docs/gene_family_scope_refactor_plan.md

Gene Family Scope Refactor Plan

本文档记录基因家族分析链路的代码通读结果、当前直接逻辑、关联代码、主要风险、修订方案和真实数据验收计划。

测试基准固定为：

• target species: Lc,Dl
• reference species: At
• comparative species: Os
• family: SPL
• real data job: job_1779115921147_d4e0a7
• primary input dir: input/job_1779115921147_d4e0a7
• primary workspace: workspace/job_1779115921147_d4e0a7

1. 总体结论

当前系统仍然以 prefix 作为几乎唯一的分析身份，例如 Multi.SPL、Lc.SPL、At.SPL。这在单物种串行分析中能工作，但在多物种、target/comparative/reference 混合分析中不够：

• prefix 只能表达一个字符串，不能表达本次分析使用了哪些 target、reference、comparative。
• 多个不同选择范围会写入同一批 final/{prefix}.* 文件。
• 下游阶段多数只读 final/{prefix}.final_members.*、final/{prefix}.tree.nwk，不知道这些文件对应哪个物种组合。
• 部分阶段会从“最新 manifest”或固定中间目录取文件，容易串用旧结果。
• 前端弹窗只在 identify 和 interspecies_synteny 上部分支持物种选择，其他阶段没有统一选择范围。

因此需要引入统一的 scope 作为一等概念。prefix 保留为兼容输出前缀，scope 表达真实分析范围。

建议 scope 字段：

{
  "family": "SPL",
  "prefix": "Multi.SPL",
  "scope_id": "SPL.target-Lc-Dl.ref-At.comp-Os",
  "target_species": ["Lc", "Dl"],
  "reference_species": ["At"],
  "comparative_species": ["Os"],
  "selected_species": ["Lc", "Dl", "At", "Os"],
  "stage_scope": "target_plus_reference_plus_comparative"
}

2. 代码链路通读记录

2.1 前端入口与弹窗

主要文件：apps/web/vue3/index.html

当前直接逻辑：

• 参数确认弹窗在 index.html:1718 附近定义。
• basic parameters 仍展示 confirmParams.prefix，index.html:1730。
• identify 独有 target/comparative/reference 展示在 index.html:1803 附近，但实际选择都写入 confirmParams.targetSpecies。
• reference_classification 只提供 referenceTargetPrefix 和 referencePrefix，index.html:1889 附近，没有 target/ref 物种勾选。
• 批量执行弹窗的 identify 物种选择在 index.html:2448 附近，仍只针对 identify。
• showConfirm() 在 index.html:5149 附近将 confirmParams.prefix = normalizedPrefix(params.prefix)。
• 只有 interspecies_synteny 调用 loadInterspeciesSpeciesOptions()，index.html:5125 附近。
• 单阶段执行时，executeSingleStageWithParams() 只对 identify 传 targetSpecies，只对 interspecies_synteny 传 targetSpecies/refSpeciesList，index.html:5809-5877。
• 普通执行 doExecute() 会把大量参数统一发送，包括 targetSpecies/refSpeciesList，index.html:6343-6404，但后端和 CLI 不一定消费。
• agent plan 展示仍打印 Prefix，index.html:7668。
• direct execution 只对 identify 和 reference_classification 做少量参数分支，index.html:7768-7794。
• 自动倒计时默认 15 秒，index.html:5191，批量倒计时也单独维护。

关联代码：

• /api/execute 请求进入 apps/web/vue3/main.py。
• /api/agent/plan 的返回被 runPlanForIntent() 消费，再进入批量执行。
• /api/stage-status 只按 job_id/stage/prefix 查状态，没有 scope。

当前风险：

• 用户选择 target/ref/comparative 后，只有少数阶段收到选择。
• reference_classification/motif/domain/structure/combo 仍按 prefix 读全局文件。
• 前端让用户理解和填写 prefix，用户负担过高。
• 倒计时太短且没有统一暂停机制。

2.2 Web API 与任务执行

主要文件：apps/web/vue3/main.py

当前直接逻辑：

• ExecuteRequest 定义 prefix 默认 "Test"，并有 targetSpecies/refSpeciesList，main.py:1353-1378。
• run_stage_sync() 解析 effective prefix 后调用 pipeline_backend.run_stage()，main.py:1604-1711。
• targetSpecies 只作为参数传给 backend；是否生效由 backend 的 stage 分支决定。
• _load_identify_species_selection() 从 config/input 推断 identify 可选物种，main.py:230。
• _infer_family_from_config_or_final() 和 _apply_inferred_family_to_intent() 尝试从 config/final 推断 family 和 prefix，main.py:157-227。
• /api/execute 会先查 stage status，仍然以 prefix 为粒度，main.py:2505-2534。
• /api/stage-status 调用 get_unified_stage_status(job_id,prefix,stage)，main.py:3060-3066。
• parse_natural_language() 仍从文本里抽 prefix 或构造 species.family，main.py:3344-3526。
• agent chat 中 explicit species 会强制改写 intent["prefix"] = f"{species}.{family}"，main.py:3769，多物种会变成 Multi.family。
• agent plan 阶段只在含 identify 且没有 targetSpecies 时补 identifySpeciesSelection，main.py:4535-4562。

关联代码：

• 实际 CLI 调度在 apps/web/pipeline_backend.py。
• 状态判断在 gfa_core/mas/tools/stage_status.py。
• 多智能体 stage 工具在 gfa_core/mas/tools/stage_tools.py。

当前风险：

• API 层没有统一 scope 请求模型。
• effective prefix 会从历史 manifest/final 自动推断，可能掩盖用户本次选择。
• status 只看 prefix，无法判断同一个 prefix 下不同物种组合是否已经完成。

2.3 后端 CLI 调度

主要文件：apps/web/pipeline_backend.py

当前直接逻辑：

• run_stage() 是所有 stage 的核心后端分发函数。
• identify 分支调用 run_identify.py，并传 --target-species，pipeline_backend.py:1149 附近。
• phylogeny 分支优先使用 final/{prefix}.final_members.pep.fa，并会去除蛋白末尾 *，pipeline_backend.py:1257-1339。
• reference_classification 分支固定读 final/{prefix}.tree.nwk，并用 reference_target_prefix or prefix.split(".")[0]，pipeline_backend.py:1366-1400。
• structure 分支固定检查 final/{prefix}.tree.nwk，并调用 run_structure.py --prefix {prefix}，pipeline_backend.py:1422-1444。
• domain 分支传 --proteins final/{prefix}.final_members.pep.fa，pipeline_backend.py:1450-1483。
• motif 分支传 --fasta final/{prefix}.final_members.pep.fa，pipeline_backend.py:1511-1534。
• pep/promoter/synteny/karyotype/expression/ppi/kaks/combo 也基本围绕 {prefix} 拼文件。
• combo 分支只检查 final/{prefix}.{stage}.manifest.json 是否存在，但 run_combo.py 内部仍可能找最新 manifest，pipeline_backend.py:3030-3059。
• genome_prepare 分支可以跑多物种准备，pipeline_backend.py:3706 之后。

关联代码：

• 传入参数来自 main.py:run_stage_sync() 和前端 index.html。
• 每个 CLI 自己还会再次自动发现文件和 manifest。

当前风险：

• backend 是 scope 断开的主要位置：参数收到了，但大多数 stage 没有用。
• 多数 stage 默认读 canonical 文件，导致同 prefix 不同分析范围互相覆盖。
• reference_classification 用字符串 target prefix，不足以表达 Lc+Dl 双目标。

2.4 意图识别、stage catalog、多智能体工具层

主要文件：

• gfa_core/mas/core/fast_intent.py
• gfa_core/mas/stage_catalog.py
• gfa_core/mas/tools/stage_tools.py

当前直接逻辑：

• fast_intent.py 用 ANALYSIS_KEYWORDS 判定阶段，fast_intent.py:9。
• 物种识别是简单 species code list，fast_intent.py 中 SPECIES_KEYWORDS。
• 返回字段主要是 stage/stages/gene_family/species/prefix，fast_intent.py:86-362。
• stage_catalog.py 定义 STAGE_DEPENDENCIES，如 combo 依赖 identify/phylogeny/motif/structure/domain，stage_catalog.py:60。
• complete_dependencies() 只按 stage 补依赖，不关心 scope，stage_catalog.py:382。
• StageToolExecutor.execute() 调用 pipeline_backend.run_stage()，并只传 targetSpecies/refSpeciesList 等已有参数，stage_tools.py:139-245。

关联代码：

• agent plan 使用这些 intent 和 stage catalog 输出计划。
• frontend runPlanForIntent() 消费 plan 并发起 batch execute。

当前风险：

• 多智能体计划没有 scope 字段，无法要求下游复用同一物种选择。
• 依赖完成判断无法区分不同 scope。
• repair/diagnosis agent 即使诊断出问题，也没有“修复 scope 输入/重建 stage input”的规范动作。

2.5 状态、manifest 校验和复现

主要文件：

• gfa_core/mas/tools/stage_status.py
• apps/cli/validate_manifest.py
• gfa_core/reproducibility_v2.py

当前直接逻辑：

• manifest_path_for() 固定返回 final/{prefix}.{stage}.manifest.json，stage_status.py:51。
• get_stage_status() 读取该 manifest 并校验 artifact 是否存在，stage_status.py:142。
• list_completed_stages() glob f"{prefix}.*.manifest.json"，stage_status.py:233-259。
• validate_manifest.py 校验 JSON schema 和 artifact 文件存在，validate_manifest.py:30-104。
• ReproducibilityManager 只以 job_id/prefix 建立复现目录和命令记录，reproducibility_v2.py:57-344。

关联代码：

• 前端 /api/stage-status、agent completed stage、batch 跳过逻辑依赖 stage_status。
• 每个 CLI 产出的 manifest 被 validate/status/repro 后续使用。

当前风险：

• 有 manifest 不代表本次选择范围正确。
• artifact 存在不代表来自正确 target/ref/comparative。
• 不同 scope 下的复现命令会混在同一个 prefix 目录。

2.6 契约与技能

主要目录：

• gfa_core/schemas/stages/
• skills/

当前直接逻辑：

• identify contract 只要求 final/{prefix}.final_members.id/fa，没有 scope。
• phylogeny contract 固定输入 final/{prefix}.final_members.pep.fa 和 identify manifest。
• motif/domain/structure/combo contract 也固定 prefix 文件和固定中间路径。
• combo contract 明确写死 motif/analysis/meme_out/meme.xml 和 domain/analysis/results.domtblout。
• skills 文档和 run.py 多数仍以 prefix 作为外部接口。

当前风险：

• 即使 CLI 修了，契约不改也会把后续开发拉回 prefix-only。
• combo contract 的固定中间路径会继续制造串结果。

3. 各分析阶段现状与关联

3.1 Identify

主要文件：apps/cli/run_identify.py

当前直接逻辑：

• 从 prefix 解析 family，或从 config.xlsx Analysis sheet 读取 family，run_identify.py:147、552、803。
• find_pep_files() 能读取 config 并按 target_species 过滤，run_identify.py:678-743。
• CLI 参数支持 --target-species，run_identify.py:1926。
• main 中读取 selected_target_species 并传给 find_pep_files()，run_identify.py:1995-1998。
• 逐物种跑 HMM/BLAST 后合并成 final/{effective_prefix}.final_members.pep.fa/id，run_identify.py:2235-2239。
• manifest 中已有 target_species 和 per-species artifacts，run_identify.py:2244-2304。
• InterProScan 是非阻塞 domain check，不是最终成员筛选器，run_identify.py:2239 后。

关联代码：

• backend identify 分支传 --target-species。
• phylogeny/motif/domain/structure 读取 identify 的 final members。
• identify_refine 会改写 final members，但没有 scope。

主要风险：

• identify 已支持 target，但没有 reference/comparative 角色定义。
• merged final members 覆盖同 prefix 的上一次选择。
• manifest 没有 scope_id 和输入 hash，不足以被下游校验。

3.2 Phylogeny

主要文件：apps/cli/run_phylogeny.py

当前直接逻辑：

• 默认读取 final/{prefix}.identify.manifest.json，run_phylogeny.py:177-215。
• 从 identify manifest 中找 final_proteins/final_members/proteins。
• 输出 final/{prefix}.pep.msa.fa、final/{prefix}.tree.nwk、manifest，run_phylogeny.py:902-973。
• CLI 有 --proteins override，run_phylogeny.py:1003-1005。
• tree plotting 的 subfamily 查找与 reference_classification 输出不完全统一。

关联代码：

• reference_classification、motif、domain、structure、combo 都读 tree。

主要风险：

• 没有 stage-level selection，无法表达 target-only、target+ref、target+comp 等不同树。
• 同 prefix 不同树会覆盖 final/{prefix}.tree.nwk。

3.3 Reference Classification

主要文件：apps/cli/run_reference_classification.py

当前直接逻辑：

• 目标树默认 final/{prefix}.tree.nwk。
• 如树中没有参考 tip，会用 target proteins + reference proteins 构建 helper joint tree。
• target 判断依赖 --target-prefix，reference 判断依赖 --reference-prefix。
• 输出 final/{prefix}.reference_classification.* 和兼容 {prefix}.geneID.subfamily。

关联代码：

• motif/domain/structure 优先读取 reference_classification.subfamily.tsv。
• combo 应使用 reference_classification 的 subfamily。

主要风险：

• 多目标物种 Lc,Dl 无法用单个 target_prefix 准确表达。
• 选择一个物种做分类时，现有逻辑可能仍用全体 tree。
• 没有 scope 校验，可能用不含 At 的树或旧 helper tree。

3.4 Motif

主要文件：apps/cli/run_motif.py

当前直接逻辑：

• prepare_directories() 固定 motif/analysis/meme_out，run_motif.py:57-61。
• 默认从 identify manifest 或 final/{prefix}.final_members.pep.fa 取蛋白，run_motif.py:128-167。
• run_meme() 检查 meme_out/meme.xml 存在就复用，run_motif.py:229-322。
• 输出 final motif 图、seqlogo、report、manifest。

关联代码：

• combo 固定找 motif/analysis/meme_out/meme.xml。

主要风险：

• 不同 scope 复用同一个 meme.xml。
• 不同参数或 FASTA 改变时没有严格 hash 判断。

3.5 Domain

主要文件：apps/cli/run_domain.py

当前直接逻辑：

• find_latest_manifest(final_dir, stage) 只按 *.{stage}.manifest.json 找最新，不带 prefix，run_domain.py:108-114。
• 默认从 identify/phylogeny latest manifest 或 final/{prefix}.* 取输入。
• run_hmmscan() 固定写 domain/analysis/results.domtblout，run_domain.py:239-286。
• manifest artifacts 记录 domtblout 路径，run_domain.py:856-864。

关联代码：

• combo 从 domain manifest 读取 domtblout。

主要风险：

• latest manifest 可能跨 prefix。
• domtblout 固定路径会被覆盖。
• 没有 scope 一致性。

3.6 Structure

主要文件：apps/cli/run_structure.py

当前直接逻辑：

• 默认 gene ID 文件为 final/{prefix}.final_members.id，run_structure.py:606。
• 默认 tree 为 final/{prefix}.tree.nwk。
• GFF3 自动解析为 {species_short}_final.gff3，其中 species_short 来自 prefix，run_structure.py:76。
• 优先读取 final/{prefix}.reference_classification.subfamily.tsv，run_structure.py:141-168。
• 输出 structure/analysis/{prefix}.structure.filtered.gff3、{prefix}.structure.tsv 和 final 图。

关联代码：

• combo 读取 structure manifest 中的 filtered gff3。

主要风险：

• Multi.SPL 无法映射到 Lc_final.gff3 和 Dl_final.gff3。
• 多物种结构需要 per-species GFF 过滤再合并或分图。
• tree tip、gene ID、GFF ID 没有统一校验。

3.7 Combo

主要文件：apps/cli/run_combo.py

当前直接逻辑：

• find_meme_xml() 优先固定 motif/analysis/meme_out/meme.xml，run_combo.py:36-39。
• find_latest_manifest(workspace, stage, prefix) 实际 glob *{stage}*.manifest.json，忽略 prefix，run_combo.py:56-62。
• phylogeny/structure/domain 都通过 latest manifest 获取，run_combo.py:166-224。
• domtblout 从 domain manifest artifact 获取。
• 输出 final/{prefix}.combo.manifest.json。

关联代码：

• backend combo 只检查 prefix manifest 存在，然后调用 CLI。

主要风险：

• 这是当前最容易串结果的阶段。
• 即使 backend 检查 prefix manifest，CLI 仍可能读取别的最新 manifest。
• meme XML 仍是固定目录。

3.8 下游阶段概览

需要后续逐个迁移：

• pep: 读 final/{prefix}.final_members.pep.fa，应改为 scope FASTA。
• promoter: 读单物种 genome/GFF/gene ID，scope 多物种时需 per-species。
• synteny: 偏单物种，backend 当前 gene ID 查找不够稳。
• interspecies_synteny: 已经有较好的 target/ref 物种选择和 config groups，可作为迁移模板。
• karyotype: 已支持按 config 物种循环，但 family ID 仍来自全局 final members。
• expression: 目前从 input/final glob gene ID，容易拿错；表达应明确单物种或显式选择表达物种。
• ppi/kaks/ortholog: 当前多依赖 prefix 和 interspecies outputs，需要等 scope 底座后再改。

4. 真实测试数据现状

已确认可用文件：

• input/job_1779115921147_d4e0a7/config.xlsx
• input/job_1779115921147_d4e0a7/Lc.pep.fasta
• input/job_1779115921147_d4e0a7/Dl.pep.fasta
• input/job_1779115921147_d4e0a7/Os.pep.fasta
• input/job_1779115921147_d4e0a7/At.SPL.pep.fa
• input/job_1779115921147_d4e0a7/At.SPL.subfamily
• input/job_1779115921147_d4e0a7/Lc_final.gff3
• input/job_1779115921147_d4e0a7/Dl_final.gff3
• input/job_1779115921147_d4e0a7/Os_final.gff3
• input/job_1779115921147_d4e0a7/Ref_combined.SPL.pep.fasta
• workspace/job_1779022508209_52539d/final/At.SPL.ncbi_fetch.protein.fa

已有历史结果：

• workspace/job_1779115921147_d4e0a7/final/Multi.SPL.genome_prepare.manifest.json
• workspace/job_1779115921147_d4e0a7/final/Target.SPL.identify.manifest.json
• workspace/job_1779115921147_d4e0a7/final/Lc.identify.manifest.json

后续真实测试不能只检查编译，必须检查：

• CLI exit code。
• manifest status == success。
• manifest scope_id 与本次选择一致。
• 输入 FASTA、tree tips、gene IDs、GFF IDs 的集合一致。
• target/reference/comparative 是否按本阶段定义参与。
• 不同 scope 的中间目录不互相覆盖。
• 故意制造 scope 不一致时，stage 必须拒绝继续。

5. 修订总体原则

1. prefix 只作为兼容输出前缀，不再承载完整分析范围。
2. 新增 scope 作为 stage 输入和 manifest 必填字段。
3. 每个阶段只读同一 scope 的上游 manifest。
4. 中间目录必须 scope 隔离。
5. 兼容旧文件名，但新逻辑优先读 scope manifest。
6. 一个阶段改完并通过真实数据测试后，再改下一个阶段。
7. 前端使用原有 Element Plus 弹窗技术，不换新框架。
8. config.xlsx 的 Analysis!Family 是 family 的优先来源；没有则让用户填写。
9. 默认示例使用 At.WRKY，不再全局默认 Ft.SPL 或 Multi.SPL。

6. 分阶段修订计划

Phase 0: Scope 底座

目标：

• 建立统一 scope 解析、写入、校验能力。
• 不改变阶段生物学结果，只先让所有阶段能接收/记录 scope。

拟修改文件：

• 新增 gfa_core/gene_family_scope.py
• 修改 apps/web/vue3/main.py
• 修改 apps/web/pipeline_backend.py
• 修改 gfa_core/mas/tools/stage_status.py
• 修改 gfa_core/reproducibility_v2.py
• 修改 gfa_core/schemas/core/workspace_contract_v1.md

核心能力：

• 从 config.xlsx 读取 family 和物种角色。
• 生成 scope_id。
• 生成 stage input manifest。
• 校验 upstream manifest 的 scope 是否匹配。
• 保留 canonical manifest，同时写 scope manifest。

真实测试：

• 在 job_1779115921147_d4e0a7 解析出 family SPL。
• 解析出 target Lc,Dl、reference At、comparative Os。
• 生成 scope JSON 后，检查所有文件存在且 hash 可计算。

通过标准：

• scope JSON 非空。
• species roles 正确。
• prefix 自动生成为 Multi.SPL 或由兼容逻辑给出，但不要求用户理解 prefix。

Phase 1: Identify

目标：

• identify 只鉴定用户选择的 target species。
• manifest 写入 scope。
• 输出 per-species artifacts 和 scope artifacts。

拟修改文件：

• apps/cli/run_identify.py
• apps/web/pipeline_backend.py
• apps/web/vue3/index.html
• gfa_core/schemas/stages/identify_contract_v1.md
• skills/gene-identify/SKILL.md

真实测试命令形态：

/home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_identify.py \
  --input-dir input/job_1779115921147_d4e0a7 \
  --workspace workspace/<test_job> \
  --prefix Multi.SPL \
  --target-species Lc,Dl \
  --required-pfams PF03110 \
  --cpu 4

通过标准：

• final members 只包含 Lc,Dl。
• 不包含 At 和 Os。
• PF03110 被记录到 parameters。
• manifest 含 scope_id、target_species=["Lc","Dl"]。

Phase 2: Phylogeny

目标：

• 支持用户选择建树范围。
• tree/msa 文件和 manifest 记录 scope。
• 不同建树范围不互相覆盖。

拟修改文件：

• apps/cli/run_phylogeny.py
• apps/web/pipeline_backend.py
• apps/web/vue3/index.html
• gfa_core/schemas/stages/phylogeny_contract_v1.md
• skills/phylogeny/SKILL.md

测试矩阵：

• target only: Lc,Dl
• target + reference: Lc,Dl,At
• target + comparative + reference: Lc,Dl,Os,At

通过标准：

• tree tips 数量等于 scope FASTA 记录数。
• manifest 的 upstream identify scope 匹配。
• 不同 scope 输出在不同中间目录。

Phase 3: Reference Classification

目标：

• 由 selected target species + reference species 驱动分类。
• 支持 Lc,Dl 多目标物种。
• At 只作为参考，不写入目标 assignment。

拟修改文件：

• apps/cli/run_reference_classification.py
• apps/web/pipeline_backend.py
• apps/web/vue3/index.html
• gfa_core/schemas/stages/reference_classification_contract_v1.md

通过标准：

• assignment/subfamily 目标行只包含 Lc,Dl。
• reference evidence 来自 At.SPL.subfamily。
• 如果 tree scope 不含 At 且需要 At，必须构建 helper tree 并记录。

Phase 4: Motif

目标：

• MEME 输出目录 scope 隔离。
• 复用 MEME 前校验 FASTA hash、参数、scope_id。

拟修改文件：

• apps/cli/run_motif.py
• apps/cli/run_combo.py
• gfa_core/schemas/stages/motif_contract_v1.md
• skills/motif-analysis/SKILL.md

通过标准：

• Lc,Dl 和 Lc 两次运行生成不同 meme_out。
• 修改 scope 后不会复用旧 meme.xml。

Phase 5: Domain

目标：

• hmmscan 输出 scope 隔离。
• 去除 latest manifest 跨 prefix 风险。

拟修改文件：

• apps/cli/run_domain.py
• apps/cli/run_combo.py
• gfa_core/schemas/stages/domain_contract_v1.md
• skills/domain-analysis/SKILL.md

通过标准：

• domain/analysis/{scope_id}/results.domtblout 存在。
• manifest domtblout path 指向 scope 目录。
• 故意放入其他 prefix manifest 时不会被读取。

Phase 6: Structure

目标：

• 多目标物种时按物种 GFF 过滤，再合并绘图输入或输出分物种结构图。
• tree tips、gene ID、GFF gene ID 做一致性检查。

拟修改文件：

• apps/cli/run_structure.py
• apps/web/pipeline_backend.py
• gfa_core/schemas/stages/structure_contract_v1.md
• skills/structure/SKILL.md

通过标准：

• Lc/Dl 的结构分别从 Lc_final.gff3、Dl_final.gff3 解析。
• 缺失 ID 报告到 metrics，不静默失败。

Phase 7: Combo

目标：

• 只合并同一 scope 的 phylogeny/motif/domain/structure。
• 不再从固定 meme_out 或 latest manifest 取文件。

拟修改文件：

• apps/cli/run_combo.py
• apps/web/pipeline_backend.py
• gfa_core/schemas/stages/combo_contract_v1.md

通过标准：

• 同 scope 正常出组合图。
• scope 不一致时明确失败并列出不一致 stage。

Phase 8: 下游阶段

迁移顺序：

1. pep
2. promoter
3. synteny
4. karyotype
5. interspecies_synteny
6. expression
7. ortholog
8. ppi
9. kaks
10. summary

每个阶段独立改、独立真实测试。

7. 前端统一方案

目标：

• prefix 不再作为主输入。
• 弹窗使用原有 Element Plus dialog/form/checkbox 技术。
• 所有基因家族阶段都显示统一的 species role selector。
• family 从 config.xlsx 自动填；没有则留空让用户填。
• 倒计时全局 40 秒，并增加暂停按钮。

前端字段：

• confirmParams.family
• confirmParams.scopeMode
• confirmParams.targetSpecies
• confirmParams.referenceSpecies
• confirmParams.comparativeSpecies
• confirmParams.prefixAdvanced
• confirmParams.countdownPaused

scopeMode 建议选项：

• target_only
• target_plus_reference
• target_plus_comparative
• target_plus_reference_plus_comparative
• custom

8. 诊断/修复 agent 后续定位

当前诊断 agent 的主要问题：

• 它看 log 和 manifest，但没有 stage-specific 可执行修复策略。
• 它不能主动修代码。
• 它没有把这次修订逻辑固化成规则。

后续需要给诊断 agent 增加“可修复动作库”：

• scope_mismatch: 建议重新生成 stage scope input，或阻止下游执行。
• wrong_manifest_scope: 指向正确 manifest 或要求重跑上游。
• fixed_intermediate_collision: 清理/隔离旧中间目录后重跑。
• missing_reference_family_pep: 从 NCBI fetch 结果或 At.SPL.pep.fa 建立兼容软链接。
• protein_stop_marker: 调用已有 clean fasta 逻辑。
• id_mismatch: 运行 ID normalize/映射检查，不直接猜测替换。

这些动作要写入代码和文档，不能只停留在聊天说明。

9. 逐项执行记录

后续每完成一个阶段，在这里追加：

Phase:
Modified files:
Real test command:
Real test data:
Result:
Manifest:
Artifacts:
Known residual risks:
Next phase gate:

当前状态：

• 文档创建完成。
• Phase 0 scope 解析模块已开始落地。

Phase: 0 Scope 底座
Modified files:
- gfa_core/gene_family_scope.py
- apps/cli/resolve_scope.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile gfa_core/gene_family_scope.py apps/cli/resolve_scope.py
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/resolve_scope.py --input-dir input/job_1779115921147_d4e0a7 --workspace workspace/job_1779115921147_d4e0a7 --prefix Multi.SPL --family SPL --target-species Lc,Dl --reference-species At --comparative-species Os --scope-mode target_plus_reference_plus_comparative
Real test data:
- input/job_1779115921147_d4e0a7/config.xlsx
- input/job_1779115921147_d4e0a7/At.SPL.pep.fa
- Lc/Dl/Os pep/gff/genome rows from config.xlsx
Result:
- scope_id = SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference_plus_comparative
- family = SPL
- target_species = Lc,Dl
- reference_species = At
- comparative_species = Os
- selected_species = Lc,Dl,At,Os
Artifacts:
- workspace/job_1779115921147_d4e0a7/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference_plus_comparative.scope.json
Known residual risks:
- At whole-genome pep/cds rows did not resolve in this job, but At.SPL family_pep resolved and is sufficient for reference classification.
- Existing stages do not yet consume scope JSON.
Next phase gate:
- Identify must accept --scope-file, default target species from scope, and write scope fields into manifest.

Phase: 1 Identify scope 接入
Modified files:
- apps/cli/run_identify.py
- apps/web/pipeline_backend.py
- apps/web/vue3/main.py
- gfa_core/mas/tools/stage_tools.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_identify.py apps/web/pipeline_backend.py apps/web/vue3/main.py gfa_core/mas/tools/stage_tools.py
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/resolve_scope.py --input-dir input/job_1779115921147_d4e0a7 --workspace workspace/codex_scope_identify_test2 --prefix Multi.SPL --family SPL --target-species Lc,Dl --reference-species At --comparative-species Os --scope-mode target_only --output workspace/codex_scope_identify_test2/final/Multi.SPL.identify_scope.json
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_identify.py --input-dir input/job_1779115921147_d4e0a7 --workspace workspace/codex_scope_identify_test2 --prefix Multi.SPL --scope-file workspace/codex_scope_identify_test2/final/Multi.SPL.identify_scope.json --required-pfams PF03110 --cpu 4 --skip-ref-warning
- Direct backend run_stage('identify') with scope_file in workspace/codex_scope_backend_identify_test.
Real test data:
- target: Lc,Dl
- reference: At
- comparative: Os
- family: SPL / PF03110
Result:
- CLI identify status = success
- CLI target_species = Lc,Dl
- CLI selected_species = Lc,Dl
- CLI final members = 55 total; Lc=33, Dl=22, At-like=0, Os-like=0
- CLI InterProScan status = success
- Backend identify status = success
- Backend final members = 59 total; Lc=37, Dl=22, At-like=0, Os-like=0
- Backend/CLI count difference is explained by parameter difference: backend default blast_evalue is 1e-5, CLI test used 1e-7.
Artifacts:
- workspace/codex_scope_identify_test2/final/Multi.SPL.identify.manifest.json
- workspace/codex_scope_identify_test2/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_only.identify.manifest.json
- workspace/codex_scope_identify_test2/final/Multi.SPL.final_members.id
- workspace/codex_scope_identify_test2/final/Multi.SPL.final_members.pep.fa
- workspace/codex_scope_backend_identify_test/final/Multi.SPL.identify.manifest.json
Known residual risks:
- Frontend does not yet generate scope JSON automatically.
- stage_status still checks prefix manifest only; scope-aware completed checks are not done.
- Downstream stages still ignore scope.
Next phase gate:
- Phylogeny must accept scope/stage FASTA and ensure tree tips match selected species.

Phase: 2 Phylogeny scope 接入
Modified files:
- apps/cli/run_phylogeny.py
- apps/web/pipeline_backend.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_phylogeny.py apps/web/pipeline_backend.py
- target-only CLI:
  /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_phylogeny.py --workspace workspace/codex_scope_phy_target_fast_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_target_fast_test/final/Multi.SPL.phylogeny_target_scope.json --cpu 4 --iqtree-params '-m LG+G -nt 4' --tree-layout rectangular
- target+reference CLI:
  /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_phylogeny.py --workspace workspace/codex_scope_phy_ref_fast_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --cpu 4 --iqtree-params '-m LG+G -nt 4' --tree-layout rectangular
- backend run_stage('phylogeny') with scope_file:
  workspace/codex_scope_backend_phy_test
Real test data:
- target-only: Lc,Dl SPL members from real identify output
- target+reference: Lc,Dl SPL members + At.SPL.pep.fa
Result:
- target-only CLI status = success
- target-only CLI scope counts = Lc 33, Dl 22
- target-only CLI scope_sequence_count = 55
- target-only CLI MSA count = 55
- target-only CLI tree tips = 55
- target+reference CLI status = success
- target+reference CLI scope counts = Lc 33, Dl 22, At 17
- target+reference CLI scope_sequence_count = 72
- target+reference CLI MSA count = 72
- target+reference CLI tree tips = 72
- backend phylogeny status = success
- backend phylogeny scope counts = Lc 33, Dl 22
- backend phylogeny MSA/tree tips = 55/55
Artifacts:
- workspace/codex_scope_phy_target_fast_test/final/Multi.SPL.phylogeny.manifest.json
- workspace/codex_scope_phy_target_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_only.phylogeny.manifest.json
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny.manifest.json
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.phylogeny.manifest.json
- workspace/codex_scope_backend_phy_test/final/Multi.SPL.phylogeny.manifest.json
Known residual risks:
- Frontend does not yet generate scope JSON automatically.
- target+comparative+reference requires Os family members. Current behavior should fail clearly unless Os has already been identified or Species.Family.pep.fa is provided.
- The backend still performs stop-marker cleanup on canonical final_members, while scope FASTA currently uses per-species artifacts directly. A later shared FASTA sanitizer should be applied to scope-built FASTA before all sequence stages.
- MFP model selection on target+reference took too long in a validation run and was stopped; fixed-model LG+G was used for scope verification. The production default remains configurable.
Next phase gate:
- Reference classification must consume selected target/reference species from scope and must not classify At as target.

Phase: 3 Reference classification scope 接入
Modified files:
- apps/cli/run_reference_classification.py
- apps/web/pipeline_backend.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_reference_classification.py apps/web/pipeline_backend.py
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_reference_classification.py --workspace workspace/codex_scope_phy_ref_fast_test --input-dir input/job_1779115921147_d4e0a7 --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --reference-subfamily input/job_1779115921147_d4e0a7/At.SPL.subfamily --reference-prefix At --overwrite-gene-subfamily
- Direct backend run_stage('reference_classification') with the same scope_file/reference_subfamily_file.
Real test data:
- Joint tree from Phase 2 target+reference: Lc,Dl + At, 72 tips
- Reference subfamily table: input/job_1779115921147_d4e0a7/At.SPL.subfamily
Result:
- CLI reference_classification status = success
- Backend reference_classification status = success
- assignment rows = 55
- assignment target composition = Lc 33, Dl 22, At 0
- subfamily rows = 72, retaining At 17 reference rows plus Lc/Dl rows for joint-tree plotting
- manifest scope_id = SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference
- manifest target_species = Lc,Dl; reference_species = At; comparative_species = Os
- scoped manifest exists.
Artifacts:
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.reference_classification.assignment.tsv
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.reference_classification.subfamily.tsv
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.reference_classification.confidence.tsv
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.reference_classification.manifest.json
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.reference_classification.manifest.json
Known residual risks:
- The internal tree script still accepts a single target_prefix and reference_prefix; the wrapper now filters assignment rows by scope target IDs, but a future cleanup should move scope awareness into scripts/reference_classification_from_tree.py directly.
- The compatibility file final/Multi.SPL.geneID.subfamily intentionally includes At reference rows when the classification scope includes reference species. Downstream plotting stages must use scope-aware filtering if they need target-only plots.
- Frontend still needs to generate/pass scope JSON from the target/comparative/reference checklist automatically.
Next phase gate:
- Structure/motif/domain/combo must consume scope/subfamily consistently and reuse the scoped reference classification outputs without rerunning unrelated species.

Phase: 4 Structure scope 接入
Modified files:
- apps/cli/run_structure.py
- apps/web/pipeline_backend.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_structure.py apps/web/pipeline_backend.py
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_structure.py --workspace workspace/codex_scope_phy_ref_fast_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --subfamily workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.reference_classification.subfamily.tsv --cpu 4
- Direct backend run_stage('structure') with the same scope_file.
Real test data:
- target GFF3 from scope: Lc=Lchinesis_genome.Chr.gff3.gz, Dl=Longan.v20190918.gff3.gz
- target IDs: final/Multi.SPL.final_members.id, 55 IDs
- tree/subfamily: Lc,Dl + At reference classification from Phase 2/3
Result:
- CLI structure status = success
- Backend structure status = success
- gzip GFF3 input is supported
- scanned GFF3 lines = 1,260,473
- filtered GFF3 rows = 353
- genes_with_structure = 55/55
- structure TSV ID check: final_ids=55, tsv_ids=55, missing=0, extra=0
- species composition in structure TSV: Lc 33, Dl 22
- structure PNG/PDF generated.
Artifacts:
- workspace/codex_scope_phy_ref_fast_test/structure/analysis/Multi.SPL.structure.filtered.gff3
- workspace/codex_scope_phy_ref_fast_test/structure/analysis/Multi.SPL.structure.tsv
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.structure.png
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.structure.pdf
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.structure.manifest.json
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.structure.manifest.json
Known residual risks:
- The structure plot uses a joint 72-tip tree but structure features only exist for target species Lc/Dl. This is intentional for target structure visualization after reference-guided classification, but downstream combo needs explicit scope-aware panel rules.
- run_structure.py still writes the first GFF3 path in the legacy cmd field for compatibility; the authoritative source list is parameters.gff3_sources.
Next phase gate:
- Motif must use scoped target proteins and scoped subfamily consistently, and must not silently include reference/comparative proteins unless the selected scope requests them.

Phase: 5 Motif scope 接入
Modified files:
- apps/cli/run_motif.py
- apps/web/pipeline_backend.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_motif.py apps/web/pipeline_backend.py
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_motif.py --workspace workspace/codex_scope_phy_ref_fast_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --subfamily workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.reference_classification.subfamily.tsv --nmotifs 3 --minw 6 --maxw 100 --cpu 4
- Direct backend run_stage('motif') with the same scope_file and motif parameters.
Real test data:
- scoped protein FASTA from phylogeny: phylogeny/analysis/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.input.pep.fa
- sequence composition: Lc,Dl target + At reference, 72 proteins
- subfamily file: final/Multi.SPL.reference_classification.subfamily.tsv
Result:
- CLI motif status = success
- Backend motif status = success
- MEME ran on 72 scoped proteins, not target-only final_members
- motif_count = 3
- total_sites = 109
- motif plot PNG/PDF generated
- scoped manifest exists
- MEME now records input FASTA hash and motif parameters in motif/analysis/meme_out/run_signature.json; existing results are reused only when the signature matches.
Artifacts:
- workspace/codex_scope_phy_ref_fast_test/motif/analysis/meme_out/meme.xml
- workspace/codex_scope_phy_ref_fast_test/motif/analysis/meme_out/run_signature.json
- workspace/codex_scope_phy_ref_fast_test/motif/analysis/Multi_SPL.motif_prot.txt
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.motif.png
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.motif.pdf
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.motif.manifest.json
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.motif.manifest.json
Known residual risks:
- The motif analysis directory is still the legacy shared path motif/analysis. The new MEME signature prevents stale MEME reuse, but future multi-scope parallel execution should use per-scope analysis subdirectories.
- If MEME reruns, the summary file is now regenerated; older outputs from pre-signature runs may be replaced during the first post-fix execution.
Next phase gate:
- Domain must use the same scoped protein FASTA/subfamily rules as motif and should avoid stale hmmscan outputs when scope or thresholds change.

Phase: 6 Domain scope 接入
Modified files:
- apps/cli/run_domain.py
- apps/web/pipeline_backend.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_domain.py apps/web/pipeline_backend.py
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_domain.py --workspace workspace/codex_scope_phy_ref_fast_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --subfamily workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.reference_classification.subfamily.tsv --evalue 1e-5 --cpu 4
- Direct backend run_stage('domain') with the same scope_file.
Real test data:
- scoped protein FASTA from phylogeny: 72 proteins, Lc/Dl target + At reference
- Pfam database: /home/xia-lab/00.share_data/hmm/Pfam-A.hmm
Result:
- CLI domain status = success
- Backend domain status = success
- hmmscan elapsed = 38.3 s in CLI test
- proteins_total = 72
- proteins_with_domains = 63
- significant_domain_hits = 84
- unique_domain_types = 8
- top domain = SBP, 50 hits
- domain PNG/PDF generated
- scoped manifest exists.
Artifacts:
- workspace/codex_scope_phy_ref_fast_test/domain/analysis/results.domtblout
- workspace/codex_scope_phy_ref_fast_test/domain/analysis/results.tblout
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.domain.png
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.domain.pdf
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.domain.manifest.json
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.domain.manifest.json
Known residual risks:
- hmmscan outputs still use the legacy shared domain/analysis/results.* filenames. This is safe for serial execution but not for two scopes running in parallel in the same workspace.
- Domain stage does not cache by input signature; it reruns hmmscan, which avoids stale output but costs time.
Next phase gate:
- Combo must read scoped manifests/artifacts from phylogeny, motif, structure and domain instead of relying on the latest manifest glob alone.

Phase: 7 Combo scope 接入
Modified files:
- apps/cli/run_combo.py
- apps/web/pipeline_backend.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_combo.py apps/web/pipeline_backend.py
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_combo.py --workspace workspace/codex_scope_phy_ref_fast_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --subfamily workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.reference_classification.subfamily.tsv --evalue 1e-5
- Direct backend run_stage('combo') with the same scope_file.
Real test data:
- scoped phylogeny, motif, structure and domain manifests from Phases 2, 5, 4 and 6
- subfamily file from Phase 3
Result:
- CLI combo status = success
- Backend combo status = success
- dependency check uses scoped manifests when scope_file is supplied
- combo PDF/PNG generated
- scoped combo manifest exists.
Artifacts:
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.combo_tree_motif_structure_domain.pdf
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.combo_tree_motif_structure_domain.png
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.combo.manifest.json
- workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.combo.manifest.json
Known residual risks:
- run_combo.py still finds meme.xml by legacy workspace search because motif manifest does not expose meme.xml as an artifact. The current test is correct for serial execution; a later cleanup should add meme_xml to motif artifacts and make combo consume it from the scoped motif manifest.
- Combo output filename remains legacy prefix-based, so concurrent scopes in one workspace can overwrite each other. Scoped manifest records which scope produced the current plot.
Next phase gate:
- Frontend/router needs to generate/pass scope_file automatically from target/reference/comparative choices; status/diagnostic/repair should become scope-aware.

Phase: 8 Frontend/API scope 自动传递与状态检查
Modified files:
- apps/web/vue3/main.py
- apps/web/vue3/index.html
- gfa_core/mas/tools/stage_status.py
- tasks.py
- docs/gene_family_scope_refactor_plan.md
Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/web/vue3/main.py gfa_core/mas/tools/stage_status.py tasks.py
- Direct API helper:
  get_gene_family_scope_options(job_id='job_1779115921147_d4e0a7', prefix='Multi.SPL')
- Direct API helper:
  _ensure_gene_family_scope_file(job_id='job_1779115921147_d4e0a7', prefix='Multi.SPL', stage='motif', target=Lc,Dl, reference=At, comparative=Os, scopeMode=target_plus_reference)
- Direct /api/execute function call:
  execute_stage(ExecuteRequest(stage='phylogeny', job_id='codex_scope_phy_ref_fast_test', prefix='Multi.SPL', scopeFile='workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json'))
- Direct /api/stage-status function call:
  get_stage_status(job_id='codex_scope_phy_ref_fast_test', stage='combo', prefix='Multi.SPL', scope_id='SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference')
Real test data:
- input/job_1779115921147_d4e0a7/config.xlsx
- workspace/codex_scope_phy_ref_fast_test scoped manifests from Phases 2-7
Result:
- /api/gene-family/scope-options returns family=SPL, target_species=Lc,Dl, reference_species=At, comparative_species from config.
- _ensure_gene_family_scope_file writes final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.scope.json.
- /api/execute with an existing scopeFile checks scoped phylogeny manifest and skips as already_completed instead of using the unscoped manifest.
- /api/stage-status with scope_id returns completed=True for scoped combo.
- Element Plus confirmation dialogs now expose analysis range as Target / Target+Reference / Target+Comparative / All, with checkboxes for target/reference/comparative species.
- Countdown defaults are now 40 s and include a "暂停倒计时" button for normal, batch and per-stage confirmation dialogs.
Known residual risks:
- The browser UI was validated by code-path inspection and backend function tests, not by an automated Playwright click-through in a live browser.
- Comparative species are listed from config; they are not auto-selected unless the selected scope mode includes comparative or upstream stage params specify them.
- Natural-language router still needs deeper prompt/rule cleanup so it consistently sets scopeMode when the user says target/comparative/reference in free text.
Next phase gate:
- Diagnostic/repair agent should get explicit gene-family repair playbooks for ID mismatches, stop codons, missing/ambiguous GFF/PEP/HMM inputs, and scoped manifest/final mismatch.

Phase: 9 Diagnostic/repair agent scope 接入与 stop-marker 真实修复
Modified files:
- gfa_core/mas/agents/diagnosis.py
- apps/cli/run_phylogeny.py
- apps/cli/run_motif.py
- apps/cli/run_domain.py
- docs/gene_family_scope_refactor_plan.md

Code relationship summary:
- tasks.py calls DiagnosisAgent.diagnose_stage_gate() before and after StageToolRegistry execution.
- diagnose_stage_gate() previously searched only final/{prefix}.{stage}.manifest.json or latest unscoped manifests, then checked legacy final_members files.
- Scoped stages now pass params.scopeFile from frontend/API/backend; DiagnosisAgent reads scope_id from scopeFile and loads final/{prefix}.{scope_id}.{stage}.manifest.json first.
- Protein-consuming stages now resolve protein input from scoped manifest parameters: protein_fasta, protein_source and scope_input_fasta.
- GFF/member stages now read target GFF paths from scopeFile when available instead of guessing Multi_final.gff3 or the first final/*.gff3.
- structure/combo no longer trigger protein stop-marker checks because they do not directly consume protein FASTA.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_motif.py apps/cli/run_domain.py apps/cli/run_phylogeny.py gfa_core/mas/agents/diagnosis.py
- Diagnosis preflight repair:
  DiagnosisAgent.diagnose_stage_gate('codex_scope_phy_ref_fast_test','motif','Multi.SPL', gate='preflight', params={'scopeFile': 'workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json'})
  DiagnosisAgent.apply_runtime_repairs(...)
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_domain.py --workspace workspace/codex_scope_phy_ref_fast_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --cpu 4
- /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_motif.py --workspace workspace/codex_scope_phy_ref_fast_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --nmotifs 3 --minw 6 --maxw 100 --cpu 4
- Postflight gate for motif/domain/structure/combo with the same scopeFile.

Real test data:
- scope: SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference
- protein input before repair: workspace/codex_scope_phy_ref_fast_test/phylogeny/analysis/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.input.pep.fa
- protein input after repair: workspace/codex_scope_phy_ref_fast_test/phylogeny/analysis/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.input.pep.clean.fa

Result:
- Diagnosis preflight detected protein_fasta_stop_markers on the real scoped FASTA.
- apply_runtime_repairs generated the clean scoped FASTA and a second preflight returned pass with no runtime repair candidates.
- domain CLI used input.pep.clean.fa, completed hmmscan successfully, and produced 72 proteins / 84 significant hits / 8 unique domains.
- motif CLI used input.pep.clean.fa, reran MEME because the input hash changed, produced 3 motifs / 109 sites, and generated PNG/PDF plus scoped manifest.
- postflight diagnosis results:
  motif pass ok []
  domain pass ok []
  structure pass ok []
  combo pass ok []

Behavioral fixes:
- Scoped manifest/final mismatch no longer causes diagnosis to inspect stale unscoped manifests first.
- Stop-marker repair is now job-local and actually affects downstream CLI inputs.
- ID/GFF low-overlap is treated as a reportable job-local diagnosis; the agent can write a diagnosis report, but it does not silently rewrite biological IDs.
- Plot-only combo no longer warns about empty metrics.

Known residual risks:
- The natural-language router still needs a separate pass so free text consistently maps to scopeMode and selected species.
- ID mismatch repair remains intentionally conservative: automatic semantic ID rewriting needs an explicit mapping rule or user confirmation.
- Motif/domain still write legacy shared analysis directories; serial real-data execution is validated, but parallel same-workspace multi-scope runs can still overwrite intermediate files.
Next phase gate:
- Router/plan/skill/contract cleanup should make scope explicit in natural-language planning, stage contracts and reproducibility records.

Phase: 10 Router/plan/skill/contract/repro scope 清理
Modified files:
- gfa_core/mas/utils/intent_parser.py
- gfa_core/mas/core/fast_intent.py
- apps/web/vue3/main.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/reproducibility_v2.py
- apps/web/pipeline_backend.py
- gfa_core/schemas/stages/identify_contract_v1.md
- gfa_core/schemas/stages/phylogeny_contract_v1.md
- gfa_core/schemas/stages/motif_contract_v1.md
- gfa_core/schemas/stages/domain_contract_v1.md
- gfa_core/schemas/stages/structure_contract_v1.md
- gfa_core/schemas/stages/combo_contract_v1.md
- skills/gene-identify/SKILL.md
- skills/phylogeny/SKILL.md
- skills/motif-analysis/SKILL.md
- skills/domain-analysis/SKILL.md
- gfa_core/mas/core/orchestrator.py
- gfa_core/mas/agents/discovery.py
- apps/web/vue3/static/conversation-sidebar.js
- gfa_core/pipeline_api_integration.py

Code relationship summary:
- Natural-language parsing now emits targetSpecies/referenceSpecies/comparativeSpecies/scopeMode instead of only prefix/species.
- /api/chat, /api/agent/chat and /api/agent/plan preserve these scope fields and put them into params for /api/execute.
- _ensure_gene_family_scope_file can now use the parsed natural-language scope fields directly to write a scope JSON from config.xlsx.
- Stage tool metadata now describes the real scoped inputs; structure uses final_members.id + target GFF3, not protein FASTA.
- ReproducibilityManager now records scope_id, scope_file and scope content in progress.json, commands.sh and provenance.json.
- pipeline_backend passes scope_file/scope_id into ReproducibilityManager.
- Stage contracts and skills now state that scope, not prefix alone, defines the analysis range.

Behavioral fixes:
- Removed runtime-path Ft.SPL defaults from orchestrator/discovery fallbacks. Generic RNA-seq fallback prefix is now "analysis"; gene-family fallback leaves family/species/prefix blank until config/user input supplies them.
- Natural-language example "目标物种Lc和Dl，参考物种At，比较物种Os，SPL基因家族成员鉴定和进化树" parses to:
  targetSpecies = Lc,Dl
  referenceSpecies = At
  comparativeSpecies = Os
  scopeMode = target_plus_reference_plus_comparative
  prefix = Multi.SPL
- Lowercase family names such as "spl" are normalized to SPL in the fast keyword intent path.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile gfa_core/mas/utils/intent_parser.py gfa_core/mas/core/fast_intent.py apps/web/vue3/main.py gfa_core/mas/tools/stage_tools.py gfa_core/reproducibility_v2.py apps/web/pipeline_backend.py gfa_core/mas/core/orchestrator.py gfa_core/mas/agents/discovery.py gfa_core/pipeline_api_integration.py gfa_core/mas/agents/result_parser.py gfa_core/mas/agents/result_parser_cumulative.py
- parse_gene_family_scope_selection / parse_intent_from_query / fast_intent_recognition on:
  "请用目标物种Lc和Dl，参考物种At，比较物种Os，进行SPL基因家族成员鉴定和进化树分析"
  "帮我执行Lc和Dl的spl基因家族鉴定"
- apps.web.vue3.main.parse_natural_language + _ensure_gene_family_scope_file with real job config:
  job_1779115921147_d4e0a7

Real test result:
- parse_natural_language returned prefix Multi.SPL and params containing targetSpecies Lc,Dl; referenceSpecies At; comparativeSpecies Os; scopeMode target_plus_reference_plus_comparative; geneFamily SPL.
- _ensure_gene_family_scope_file wrote:
  workspace/job_1779115921147_d4e0a7/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference_plus_comparative.scope.json
- ReproducibilityManager loaded the existing real scope file and resolved:
  scope_id = SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference
  target_species = Lc,Dl
  reference_species = At
  comparative_species = Os

Known residual risks:
- Historical docs, backups and test fixtures still contain Ft.SPL as historical data. Active runtime fallbacks were cleaned; historical records were not rewritten.
- Full browser click-through of /api/agent/plan -> confirmation dialog -> /api/execute is still not automated by Playwright.
- Some older result interpretation text still uses prefix as prose context; it no longer controls execution scope, but future interpretation cleanup should display scope_id/species sets to users.
Next phase gate:
- Update result interpretation/reporting so summaries display family + scope species sets rather than relying on prefix wording.

Phase: 11 Result interpretation/report scope 化
Modified files:
- gfa_core/mas/agents/result_parser.py
- gfa_core/mas/agents/interpretation_evidence.py
- gfa_core/mas/agents/result_parser_cumulative.py

Code relationship summary:
- ResultParserAgent now extracts scope context from manifest top-level fields, parameters or scope blocks.
- Single-stage reports now include an explicit "分析范围" section with family, scope_id, scope_mode, target/reference/comparative/selected species.
- Prompt metric bundles and LLM digests now carry scope, so downstream interpretation can cite the real analysis range.
- Cumulative reports now accept optional scope_id, auto-discover a scope_id from matching manifests when omitted, and prefer scoped manifests before legacy manifests.
- Evidence-first cumulative interpretation now filters manifests by scope_id and uses the scope label in report title/biological story.
- The fallback cumulative mixin was updated to keep the same scope fields, avoiding a future import path reverting to prefix-only wording.

Behavioral fixes:
- "Multi.SPL" is no longer presented as if it were the biological species range in reports.
- Real scope "SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference" reports as:
  family = SPL
  target_species = Lc,Dl
  reference_species = At
  comparative_species = Os
  selected_species = Lc,Dl,At
- Cumulative interpretation can now distinguish scoped manifests from legacy manifests for the same prefix.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile gfa_core/mas/agents/result_parser.py gfa_core/mas/agents/result_parser_cumulative.py gfa_core/mas/agents/interpretation_evidence.py
- ResultParserAgent._parse_result and _generate_report using:
  workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.domain.manifest.json
- ResultParserAgent.generate_cumulative_report using real completed stages:
  phylogeny,motif,domain,structure,combo

Real test result:
- Parsed scope label:
  SPL家族；目标物种=Lc,Dl；参考物种=At；比较物种=Os
- Single-stage markdown includes:
  基因家族 SPL, Scope ID SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference, 目标物种 Lc/Dl, 参考物种 At, 比较物种 Os, 实际参与分析物种 Lc/Dl/At.
- Cumulative report auto-discovered scope_id when scope_id was omitted and loaded the scoped manifests.

Known residual risks:
- Evidence helper sibling-manifest lookup still falls back to legacy sibling names for some cross-stage raw-file parsing; scoped report selection works, but sibling evidence enrichment can be made fully scope-aware later.
Next phase gate:
- Continue with the next analysis stage or reporting polish pass; prioritize any active user-facing confusion before broad refactors.

Phase: 11b Evidence report hardcoded-prefix cleanup
Modified files:
- gfa_core/mas/agents/interpretation_evidence.py
- gfa_core/mas/agents/result_parser.py

Behavioral fixes:
- Removed active result-report hardcoded Ft.SPL/Ath SPL wording from synteny, interspecies_synteny, PPI and publication-claim templates.
- Evidence report text now uses the current scope label/family label for target/reference family statements.
- Identify reference-count fallback no longer only recognizes SPL_Ath.fasta; it accepts reference-role, known-family or family-named inputs from manifest.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile gfa_core/mas/agents/result_parser.py gfa_core/mas/agents/result_parser_cumulative.py gfa_core/mas/agents/interpretation_evidence.py
- ResultParserAgent.generate_cumulative_report with real scope job codex_scope_phy_ref_fast_test.

Real test result:
- Cumulative report scope label is:
  SPL家族；目标物种=Lc,Dl；参考物种=At；比较物种=Os
- Serialized cumulative report no longer contains active "Ft.SPL" or "Ath SPL" wording.

Phase: 12 Promoter scope guard and single-target resolution
Modified files:
- apps/cli/run_promoter.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/promoter_contract_v1.md

Code relationship summary:
- run_promoter.py now accepts --scope-file.
- In scope mode, promoter resolves GFF/GFF3 and genome FASTA from the single selected target species instead of guessing from prefix.
- Promoter refuses multi-target scope because promoter extraction cannot safely merge different genomes/GFFs in one run.
- Success and failure manifests now record scope_id, scope_mode, scope_file, target/reference/comparative/selected species and actual gff3/genome/gene_id files.
- pipeline_backend passes scope_file to run_promoter.py and skips legacy "first GFF/first genome" selection when scope is available.
- Stage metadata and contract now state that promoter is a single-target stage.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_promoter.py apps/web/pipeline_backend.py gfa_core/mas/tools/stage_tools.py gfa_core/mas/agents/interpretation_evidence.py
- Multi-target guard test:
  /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_promoter.py --input-dir input/job_1779115921147_d4e0a7 --workspace workspace/codex_promoter_scope_guard_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --plantcare-tab workspace/codex_scope_phy_ref_fast_test/promoter/analysis/nonexistent.tab
- Single-target real scope resolution:
  build_gene_family_scope(... target_species=['Lc'] ...) + resolve_promoter_scope_inputs()

Real test result:
- Multi-target Lc,Dl scope failed early with clear message:
  Promoter 阶段一次只能分析一个目标物种；当前 scope target_species=['Lc', 'Dl']。请在弹窗中只选择一个 target，或分别运行每个目标物种。
- Failure manifest was written to:
  workspace/codex_promoter_scope_guard_test/final/Multi.SPL.promoter.manifest.json
  and contains scope_id + target_species.
- Single-target Lc scope resolved real files:
  /home/xia-lab/wutx/plantAnalysisAgent/ref_data/lizhi_longyan/Lchinesis_genome.Chr.gff3.gz
  /home/xia-lab/wutx/plantAnalysisAgent/ref_data/lizhi_longyan/Lchinesis_genome.Chr.fasta.gz

Known residual risks:
- Full PlantCARE execution was not run because it depends on external web/email service. Scope input resolution and multi-target guard were validated with real project data.
- Promoter still writes legacy output names for compatibility; scoped manifest is written in addition when execution reaches success.
Next phase gate:
- Move to synteny/interspecies_synteny, which need careful target/comparative/reference separation and should not infer species from Multi.<family>.

Phase: 13 Synteny scope guard and full-proteome target resolution
Modified files:
- apps/cli/run_synteny.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/synteny_contract_v1.md

Code relationship summary:
- run_synteny.py now accepts --scope-file.
- In scope mode, synteny resolves target pep/GFF/genome from the single selected target species.
- Synteny refuses multi-target scope because it is an intraspecies MCScanX stage and cannot safely combine multiple genomes in one run.
- validate_inputs now prioritizes full target proteome over identify final family proteins; identify proteins are only a legacy fallback.
- final/{prefix}.final_members.id is now used as the preferred family gene list for Circos filtering.
- Success and failure manifests now record scope fields and write a scoped manifest copy when scope_id is present.
- pipeline_backend passes scope_file to run_synteny.py and skips legacy "first GFF/first genome" selection when scope is available.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_synteny.py apps/web/pipeline_backend.py gfa_core/mas/tools/stage_tools.py
- Multi-target guard test:
  /home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/run_synteny.py --input-dir input/job_1779115921147_d4e0a7 --workspace workspace/codex_synteny_scope_guard_test --prefix Multi.SPL --scope-file workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json --cpu 1
- Single-target real scope resolution:
  resolve_synteny_scope_inputs(workspace/codex_promoter_scope_guard_test/final/Multi.SPL.Lc.promoter.scope.json)

Real test result:
- Multi-target Lc,Dl scope failed early with clear message:
  Synteny 阶段是种内共线性，一次只能分析一个目标物种；当前 scope target_species=['Lc', 'Dl']。请在弹窗中只选择一个 target，或分别运行每个目标物种。
- Failure manifest was written to:
  workspace/codex_synteny_scope_guard_test/final/Multi.SPL.synteny.manifest.json
  and contains scope_id + target_species.
- Single-target Lc scope resolved real files:
  /home/xia-lab/wutx/plantAnalysisAgent/ref_data/lizhi_longyan/Lchinesis_genome.pep.sim.fa.gz
  /home/xia-lab/wutx/plantAnalysisAgent/ref_data/lizhi_longyan/Lchinesis_genome.Chr.gff3.gz
  /home/xia-lab/wutx/plantAnalysisAgent/ref_data/lizhi_longyan/Lchinesis_genome.Chr.fasta.gz

Known residual risks:
- Full MCScanX execution was not run in this pass because it can be long and depends on local tool availability/data scale. Scope input resolution and multi-target guard were validated with real project data.
- Some legacy behavior still allows identify final proteins as fallback when no full target proteome is found; this is retained for compatibility but is not the recommended path.
Next phase gate:
- Move to interspecies_synteny. It should use target/reference/comparative roles explicitly rather than scanning arbitrary protein files.

Phase: 14 Interspecies synteny scope selection and gzip-safe real data discovery
Modified files:
- apps/cli/run_interspecies_synteny.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/interspecies_synteny_contract_v1.md

Code relationship summary:
- run_interspecies_synteny.py now accepts --scope-file.
- Scope selection is applied before discovery:
  target_species -> target side
  selected reference/comparative species -> ref side
- In scope mode, species files are resolved from scope files first.
- In scope mode, full proteome can only come from scope `pep`; family_pep is not treated as all_pep.
- reference/comparative family_pep can still be used for the interspecies family tree.
- gzip FASTA/GFF files are now read through open_text(), which supports .gz ref_data paths.
- Success/failure manifests now record scope_id, scope_mode, scope_file, target/reference/comparative/selected species and write a scoped manifest copy when scope_id is present.
- pipeline_backend passes scope_file to run_interspecies_synteny.py and skips legacy input pep precheck when scope is available.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_interspecies_synteny.py apps/web/pipeline_backend.py gfa_core/mas/tools/stage_tools.py
- Real scope discovery with:
  workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json
- Real target+reference+comparative discovery with:
  workspace/job_1779115921147_d4e0a7/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference_plus_comparative.scope.json
- Failure manifest write test with real scope metadata.

Real test result:
- target_plus_reference scope selected:
  targets = Lc,Dl
  refs = At
  target_all_peps = Lc,Dl
  ref_all_peps = empty because At all-pep path is missing in scope
  ref_family_protein = generated selected_ref.family.pep.fa
- target_plus_reference_plus_comparative scope selected:
  targets = Lc,Dl
  refs = At,Os
  target_all_peps = Lc,Dl
  ref_all_peps = Os only
  ref_gff = At,Os
- Failure manifest records:
  scope_id = SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference
  target_species = Lc,Dl
  reference_species = At
  comparative_species = Os

Known residual risks:
- Full MAFFT/IQ-TREE/MCScanX execution was not run in this pass because it can be long. The corrected scope selection, gzip-safe file discovery and manifest behavior were validated with real project data.
- At whole-proteome path is absent in the current scope file, so interspecies MCScanX for At should remain partial/skipped unless the At all-pep file is added or genome_prepare provides it. This is correct behavior; the system must not substitute At.SPL.pep.fa as whole proteome.
Next phase gate:
- Move to karyotype, which is another target-side stage and must not run all targets together unless explicitly designed for multi-target output.

Phase: 15 Karyotype target-scope selection and gzip GFF support
Modified files:
- apps/cli/run_karyotype.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/karyotype_contract_v1.md

Code relationship summary:
- run_karyotype.py now accepts --scope-file.
- In scope mode, karyotype uses only `scope_file.target_species` and resolves each target species' GFF/genome from scope.
- Multi-target scope remains supported, but output is per target species (`{Species}.{Family}.karyotype.*`), not one merged cross-species karyotype.
- gzip GFF files are now read through open_text(), supporting real ref_data *.gff3.gz inputs.
- Success/failure manifests now record scope_id, scope_mode, scope_file, target/reference/comparative/selected species and write a scoped manifest copy when scope_id is present.
- pipeline_backend passes scope_file to run_karyotype.py.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_karyotype.py apps/web/pipeline_backend.py gfa_core/mas/tools/stage_tools.py
- discover_species_inputs() using real scope:
  workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json
- build_simplified_gff() against real gzip GFFs for Dl and Lc using real final_members IDs.
- write_manifest() scope-copy test.

Real test result:
- Scope discovery selected only target species:
  Lc, Dl
- Resolved real files:
  Lc: Lchinesis_genome.Chr.gff3.gz + Lchinesis_genome.Chr.fasta.gz
  Dl: Longan.v20190918.gff3.gz + Longan_Chr.v20190918.fasta.gz
- GFF matching succeeded on gzip inputs:
  Dl sample: 5/5 matched rows
  Lc sample: 5/5 matched rows
- Manifest writes both:

```text
Phase: 16 PEP scoped protein-set input
Modified files:
- apps/cli/run_pep.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/pep_contract_v1.md

Code relationship summary:
- run_pep.py now accepts --scope-file and --fasta.
- In scope mode, PEP resolves the same protein FASTA used by the matching phylogeny scope manifest (`parameters.scope_input_fasta`), then removes terminal/internal `*` into a job-local clean FASTA when needed.
- target_only scope may fall back to `final/{prefix}.final_members.pep.fa`; multi-species/reference/comparative scopes must use the phylogeny-scoped merged FASTA unless --fasta is explicitly provided.
- Success manifests record scope_id, scope_mode, scope_file, target/reference/comparative/selected species, family, and write a scoped manifest copy.
- pipeline_backend passes the scoped FASTA into the existing PEP skill path and annotates the copied manifest with scope metadata, so the current faster skill implementation remains compatible.

Real test command:
- Compile run_pep.py / pipeline_backend.py / stage_tools.py.
- Resolve PEP input from the real target+reference scope and verify it points to the phylogeny-scoped clean FASTA.
- Run the PEP skill on that real scoped FASTA and verify final manifest contains scope metadata and the expected protein count.

Real test result:
- py_compile passed.
- CLI resolver:
  `/home/xia-lab/wutx/00test/bbav5_0502_Yang_Tihuan.7880/workspace/codex_scope_phy_ref_fast_test/phylogeny/analysis/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.input.pep.clean.fa`
- Backend resolver uses the same scoped clean FASTA instead of `final/{prefix}.final_members.pep.clean.fa`, preventing cross-scope overwrite.
- PEP skill loaded 72 real sequences and produced:
  `workspace/codex_pep_scope_fast_test/final/Multi.SPL.pep.manifest.json`
  `workspace/codex_pep_scope_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.pep.manifest.json`
- Manifest contains `scope_id=SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference` in both top-level fields and `parameters`.

Phase: 17 Expression single-target scope selection
Modified files:
- apps/cli/run_expression.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/expression_contract_v1.md

Code relationship summary:
- Expression is treated as a single-target-species stage because expression matrices are species-specific.
- pipeline_backend now reads scope_file for expression and rejects multi-target scopes before execution.
- In scope mode, backend resolves member IDs from `identify/{Species}/final/{Species}.final_gfmembers.id` instead of globbing any `*geneID*` or multi-species `final/{prefix}.final_members.id`.
- Removed the duplicated expression/gene/sample discovery block that could overwrite the first resolved inputs.
- run_expression.py accepts `--scope-file`, validates that exactly one target species is selected, allows the scoped per-species gene ID file to satisfy upstream validation, records scope fields in manifest, and writes a scoped manifest copy.
- comparative_summary.target_species now uses the scoped target species instead of parsing `Multi` from `Multi.SPL`.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_expression.py apps/web/pipeline_backend.py gfa_core/mas/tools/stage_tools.py
- Resolver test against real Lc/Dl identify outputs from `workspace/codex_scope_phy_ref_fast_test/identify/{Species}/final/{Species}.final_gfmembers.id`.
- Full expression run with a small real Lc member subset expression matrix and `rnaR` Rscript.
- Multi-target guard run with target_species=[Lc,Dl].

Real test result:
- py_compile passed.
- Resolver selected:
  `workspace/codex_scope_phy_ref_fast_test/identify/Lc/final/Lc.final_gfmembers.id`
  `workspace/codex_scope_phy_ref_fast_test/identify/Dl/final/Dl.final_gfmembers.id`
- Single-target Lc expression run succeeded in `workspace/codex_expression_scope_fast_test`, using:
  `workspace/codex_expression_scope_fast_test/identify/Lc/final/Lc.final_gfmembers.id`
- R generated heatmap/correlation/barplot PDF+PNG and expression/correlation txt outputs.
- Manifest contains `scope_id=SPL.target-Lc.ref-At.comp-Os.target_only`, `gene_count=33`, and comparative_summary.target_species=`Lc`.
- Multi-target scope failed early with the intended message:
  `Expression 阶段一次只能分析一个目标物种；当前 scope target_species=['Lc', 'Dl']。请在弹窗中只选择一个 target，或分别运行每个目标物种。`

Phase: 18 Ortholog target-to-reference scope mapping
Modified files:
- apps/cli/run_ortholog.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/ortholog_contract_v1.md

Code relationship summary:
- run_ortholog.py had the broken inline CONDA_BIN statement repaired.
- pipeline_backend ortholog stage now reads scope_file when provided.
- In scope mode, target family proteins are resolved per species from:
  `identify/{Species}/final/{Species}.final_gfmembers.pep.fa`, then compatible `{Species}.{Family}.pep.fa` fallbacks.
- Reference protein resolves from scope `family_pep` first, then compatible `{Reference}.{Family}.pep.fa`; it no longer depends only on hardcoded Ath/At filenames.
- Multi-target scope runs one BLASTP job per target species against the first reference species, then merges mappings.
- Output mapping now includes source/target species columns:
  `Source_ID Target_ID Identity(%) Source_Species Target_Species`
- Manifest records scope fields and writes a scoped manifest copy.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_ortholog.py apps/web/pipeline_backend.py gfa_core/mas/tools/stage_tools.py
- run_stage('ortholog') with real scope:
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json`
  target: Lc,Dl; reference: At; family: SPL.

Real test result:
- py_compile passed.
- BLASTP 2.17.0+ was available.
- Ortholog ran two real BLASTP jobs:
  Lc final family proteins vs `input/job_1779115921147_d4e0a7/At.SPL.pep.fa`
  Dl final family proteins vs `input/job_1779115921147_d4e0a7/At.SPL.pep.fa`
- Produced 54 mapped pairs.
- Output:
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.ortholog_pairs.txt`
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.orthologs.txt`
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.ortholog_pairs.raw.txt`
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.ortholog_blastp.txt`
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.ortholog.manifest.json`
- Example mapping:
  `LITCHI014255.m1    At2g47070    49.81    Lc    At`

Phase: 19 PPI scoped ortholog mapping input
Modified files:
- apps/cli/run_ppi.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/ppi_contract_v1.md

Code relationship summary:
- run_ppi.py had the broken inline CONDA_BIN statement repaired.
- pipeline_backend PPI stage now reads scope_file when provided.
- In scope mode, reference_species is converted to the STRING species code (`At -> ath`, `Os -> osa`).
- If an ortholog mapping exists, PPI builds a scoped source gene ID file from `{prefix}.orthologs.txt` / `{prefix}.ortholog_pairs.txt` and passes the mapping to the existing PPI skill.
- Multi-target scope now requires Ortholog first; PPI will not send mixed local IDs from multiple species directly to STRING.
- PPI manifest annotation uses the shared scope manifest helper and writes a scoped manifest copy after a real PPI run.

Real-data validation command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_ppi.py apps/web/pipeline_backend.py gfa_core/mas/tools/stage_tools.py
- build_ppi_gene_ids_from_ortholog_mapping() with real ortholog output:
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.orthologs.txt`
- skills.ppi.run.parse_ortholog_mapping() on the same real mapping.

Real-data validation result:
- py_compile passed.
- From real Lc/Dl -> At ortholog mapping, PPI source ID extraction produced 54 source genes:
  `workspace/codex_scope_phy_ref_fast_test/ppi/analysis/codex_ppi_source_ids.txt`
- STRING species code resolution:
  `At -> ath`, `Os -> osa`
- Existing PPI skill parsed 54 source→At mappings, e.g.:
  `LITCHI014255.m1 -> At2g47070`

Known residual risk:
- Full STRING network querying was not executed in this pass because it depends on external network service availability. The scoped input/mapping path was validated with real ortholog data; a complete online PPI run should be verified separately when network access is allowed.

Phase: 20 Ka/Ks scoped target-reference runs and YN00 parser correction
Modified files:
- apps/cli/run_kaks_v2.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/kaks_contract_v1.md

Code relationship summary:
- run_kaks_v2.py now has a valid CONDA_BIN/PATH block.
- run_kaks_v2.py can read gzipped FASTA inputs.
- run_kaks_v2.py now matches IDs case-insensitively after removing transcript/version suffixes, so `At2g47070` can match `AT2G47070.1`.
- run_kaks_v2.py now detects pal2nal error output instead of passing an empty/error codon alignment into yn00.
- YN00 parsing was corrected. The old parser incorrectly treated columns `S/N/t` as `Ka/Ks/KaKs`; the corrected parser reads `dN/dS/omega`.
- pipeline_backend KaKs stage now reads scope_file.
- In scope mode, KaKs splits multi-target scopes into per-target runs:
  `{prefix}.{Target}_to_{Reference}`
- Target CDS/PEP and reference CDS/PEP are resolved per species. Reference PEP is intentionally resolved from the same whole-genome annotation source as reference CDS, not from `{Reference}.{Family}.pep.fa`, because Ka/Ks requires CDS and protein from the same transcript set.
- Aggregate and scoped manifests are written:
  `{prefix}.kaks.manifest.json`
  `{prefix}.{scope_id}.kaks.manifest.json`

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_kaks_v2.py apps/web/pipeline_backend.py gfa_core/mas/tools/stage_tools.py
- Resolve real scope files:
  Lc CDS: Lchinesis_genome.sim.cds.fa.gz
  Dl CDS: longan.v20190918.sim.CDS.gz
  At CDS: Arabidopsis_thaliana.TAIR10.cds.all.fa
  At PEP: Arabidopsis_thaliana.TAIR10.pep.all.fa
- Split real ortholog mapping:
  Lc->At: 33 pairs
  Dl->At: 21 pairs
- Run full scoped KaKs through run_stage('kaks').

Real test result:
- py_compile passed.
- Initial test with `At.SPL.pep.fa` failed pal2nal consistency checks, proving that NCBI/downloaded family PEP cannot be mixed with TAIR10 CDS for Ka/Ks.
- After switching reference PEP to TAIR10 whole PEP and fixing YN00 parser, full run succeeded:
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.Lc_to_At.kaks_results.csv`
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.Dl_to_At.kaks_results.csv`
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.kaks.manifest.json`
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference.kaks.manifest.json`
- Successful pairs:
  Lc->At: 33/33
  Dl->At: 21/21
- Corrected example values:
  `LITCHI014255.m1, At2g47070, Ka=0.3281, Ks=1.3925, Ka/Ks=0.2356`
  `Dil.01g033640.1.t1, At5g18830, Ka=0.3490, Ks=1.5298, Ka/Ks=0.2282`
- Aggregate manifest records:
  `species_runs=2`, `input_pairs=54`, `scope_id=SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference`.

Phase: 21 Summary scoped objective result index
Modified files:
- apps/cli/run_summary.py
- apps/web/pipeline_backend.py
- gfa_core/mas/tools/stage_tools.py
- gfa_core/schemas/stages/summary_contract_v1.md

Code relationship summary:
- run_summary.py was replaced with a self-contained manifest aggregator. The old file imported a missing summary_agent module and could not run as a reliable CLI stage.
- Summary is defined as an objective result index and QC report, not deep biological interpretation.
- Summary scans `final/*.manifest.json`, excludes summary/minimal self manifests, filters child manifests whose prefix does not match the requested prefix, groups results by `scope_id`, and records stage status, key metrics, artifacts and consistency checks.
- Summary supports gene-family and RNA-seq style manifests because it only depends on the shared manifest schema fields.
- In scope mode, Summary writes both a global summary and a scoped summary:
  `{prefix}.summary_report.md`
  `{prefix}.summary.files.tsv`
  `{prefix}.summary.manifest.json`
  `{prefix}.{scope_id}.summary_report.md`
  `{prefix}.{scope_id}.summary.files.tsv`
  `{prefix}.{scope_id}.summary.manifest.json`
- pipeline_backend now runs Summary through `apps/cli/run_summary.py` and no longer requires an identify manifest as the seed.

Real test command:
- /home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_summary.py
- Run CLI summary with real scope:
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.phylogeny_ref_scope.json`
- Run backend `run_stage('summary')` on the same real workspace.
- Run CLI summary on an existing real RNA-seq manifest workspace:
  `workspace/job_1778662928678_06813e`, prefix `Os`.

Real test result:
- py_compile passed.
- CLI and backend Summary both succeeded.
- Summary grouped the real Lc,Dl target / At reference / Os comparative SPL run into:
  `SPL.target-Lc-Dl.ref-At.comp-Os.target_only`: identify
  `SPL.target-Lc-Dl.ref-At.comp-Os.target_plus_reference`: combo, domain, kaks, motif, ortholog, phylogeny, reference_classification, structure
- Output report:
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.summary_report.md`
- Output artifact index:
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.summary.files.tsv`
- Output manifest:
  `workspace/codex_scope_phy_ref_fast_test/final/Multi.SPL.summary.manifest.json`
- Aggregate manifest records:
  `manifest_count=9`, `scope_count=2`, `artifact_count=103`.
- RNA-seq manifest aggregation also succeeded:
  `workspace/job_1778662928678_06813e/final/Os.summary_report.md`
  `workspace/job_1778662928678_06813e/final/Os.summary.files.tsv`
  `workspace/job_1778662928678_06813e/final/Os.summary.manifest.json`
- RNA-seq summary grouped 13 unscoped stage manifests, including:
  annotation, config_autofill, rnaseq_align, rnaseq_candidate_family, rnaseq_de, rnaseq_fgsea, rnaseq_go_enrich, rnaseq_kegg_enrich, rnaseq_qc, rnaseq_quant, rnaseq_sample_qc, rnaseq_wgcna, samples_autofill.
- RNA-seq aggregate manifest records:
  `manifest_count=13`, `scope_count=1`, `artifact_count=305`.

Known residual risks:
- Summary currently reports objective artifacts/metrics/status only. It intentionally does not infer biological conclusions or recommend subfamilies; that should be a separate deep interpretation stage built on top of these indexed outputs.

Phase: 22 Full real-data end-to-end validation and fixes
Modified files:
- apps/web/pipeline_backend.py
- apps/cli/run_reference_classification.py
- apps/cli/run_promoter.py
- apps/cli/run_synteny.py

Validation workspace:
- workspace/codex_full_flow_real_20260519_235947

Real data:
- target species: Lc,Dl
- reference species: At
- comparative species: Os
- family: SPL
- input dir: input/job_1779115921147_d4e0a7

Issues found and fixed:
- Phylogeny advanced IQ-TREE params beginning with `-` were passed as a separate argv value, so argparse treated the value as another option. pipeline_backend now passes these as `--iqtree-params=<value>` and `--mafft-params=<value>`.
- reference_classification auto-selected `Target.SPL.geneID.subfamily`, which already contained target labels, instead of the pure At reference file. It now prefers `{reference_prefix}.{family}.subfamily`, for example `At.SPL.subfamily`.
- promoter failed on non-UTF-8 stderr from external tools. The command runner now decodes subprocess output with replacement instead of raising UnicodeDecodeError.
- synteny failed or fell back to very slow BLASTP because whole-proteome FASTA files contained terminal `*`/`.` stop markers and DIAMOND v2 uses `--threads`, not `--thread`. Synteny now writes a BLAST-safe cleaned protein FASTA and uses the correct DIAMOND option.

Final validation result:
- Final summary report:
  `workspace/codex_full_flow_real_20260519_235947/final/Multi.SPL.summary_report.md`
- Final summary manifest:
  `workspace/codex_full_flow_real_20260519_235947/final/Multi.SPL.summary.manifest.json`
- Final summary artifact index:
  `workspace/codex_full_flow_real_20260519_235947/final/Multi.SPL.summary.files.tsv`
- Summary recorded:
  `manifest_count=18`, `scope_count=3`, `artifact_count=181`
- Final consistency check:
  `No manifest-level consistency issues detected`

Successful final stages:
- target_plus_reference scope:
  identify, phylogeny, reference_classification, structure, domain, motif, pep, combo, karyotype, interspecies_synteny, ortholog, ppi, kaks
- Lc target_only scope:
  expression, promoter, synteny
- Dl target_only scope:
  promoter, synteny

Key final metrics:
- Identify/phylogeny protein set: 76 sequences = 59 target + 17 At reference.
- Reference classification: target_gene_count=59, reference_gene_count=17.
- Domain: proteins_total=76, proteins_with_domains=67, domain_coverage_rate=0.8816.
- Motif: sequence_count=76.
- Ortholog: ortholog_pairs=58.
- Ka/Ks: species_runs=2, input_pairs=58.
- Karyotype: species_count=2, success_count=2.
- Interspecies synteny: status=success, Lc.At and Dl.At collinearity files produced.
- Promoter:
  Lc and Dl target_only runs both succeeded.
- Synteny:
  Lc gene_count=31896, Dl gene_count=37142.

Known residual risks:
- The validation used a generated full_clean scope for interspecies_synteny because the original scope had empty At whole-proteome paths. The next front-end/config revision should ensure whole-proteome reference paths are filled in the generated scope by default.
- Full phylogeny with ModelFinder is slow for validation. A future UI/plan option should expose fast verification mode such as `-m LG+G4 -nt <cpu> -fast`.

Phase: 23 Automatic scope artifact and reference-path repair
Modified files:
- apps/cli/run_promoter.py
- apps/cli/run_synteny.py
- gfa_core/gene_family_scope.py

Purpose:
- Remove manual intervention for two remaining scope problems:
  1. Single-target stages wrote scope manifests but their artifacts still pointed to generic files, so Lc/Dl runs could overwrite each other.
  2. Generated scopes could leave reference whole-proteome paths empty when config.xlsx recorded files stored in local ref_data instead of input/final.

Fixes:
- promoter and synteny now materialize scope-specific artifact copies when `scope_id` is present. Generic legacy outputs are still written for compatibility, but scoped manifests point to stable paths containing the sanitized scope ID.
- promoter failure handling now writes both generic and scoped failure manifests when a scope file is provided.
- promoter PATH initialization was repaired so the test4py311 tool directory is actually prepended; this fixed a real `gffread` not found failure.
- gene_family_scope now searches local reference-data roots, including `/home/xia-lab/wutx/plantAnalysisAgent/ref_data`, by the config original file name when input/workspace/final lookup fails.

Real-data validation:
- Scope generation:
  `/tmp/codex_scope_autofill_test.json`
  resolved At whole-genome gff/cds/genome/pep paths from local ref_data, including:
  `/home/xia-lab/wutx/plantAnalysisAgent/ref_data/ath/Arabidopsis_thaliana.TAIR10.pep.all.fa`
- Synteny rerun with real Lc and Dl target_only scopes:
  `Multi.SPL.SPL.target-Lc.ref-At.comp-Os.target_only.synteny.manifest.json`
  `Multi.SPL.SPL.target-Dl.ref-At.comp-Os.target_only.synteny.manifest.json`
  Both scoped manifests point to scope-specific blast/circos/report artifacts and `missing=0` for registered artifact paths.
- Promoter rerun with real Lc and Dl target_only scopes using existing local PlantCARE tab:
  `Multi.SPL.SPL.target-Lc.ref-At.comp-Os.target_only.promoter.manifest.json`
  `Multi.SPL.SPL.target-Dl.ref-At.comp-Os.target_only.promoter.manifest.json`
  Both scoped manifests point to scope-specific report and PlantCARE/table artifacts and `missing=0` for registered artifact paths.

Validation commands:
- `/home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile apps/cli/run_promoter.py apps/cli/run_synteny.py gfa_core/gene_family_scope.py`
- `/home/xia-lab/miniconda3/envs/test4py311/bin/python apps/cli/resolve_scope.py --input-dir input/job_1779115921147_d4e0a7 --workspace workspace/codex_full_flow_real_20260519_235947 --prefix Multi.SPL --family SPL --target-species Lc,Dl --reference-species At --comparative-species Os --scope-mode target_plus_reference --output /tmp/codex_scope_autofill_test.json`
- Real Lc/Dl synteny and promoter reruns in `workspace/codex_full_flow_real_20260519_235947`.

Known residual risks:
- The promoter validation intentionally used an existing local PlantCARE tab to avoid submitting to the external PlantCARE service. That tab was empty after filtering, so R promoter visualizations reported "no lines available in input"; the stage still completed and scoped artifact/manifest behavior was validated.

2026-05-26 追加记录：Stage Requirements 与 Agent 职责边界

KaKs/PPI/scope 讨论暴露出 PlanningAgent、DiagnosisAgent、ResultParserAgent 与执行层之间存在职责混杂：静态依赖、动态需求、artifact 解析、完成状态、修复策略在多个文件重复实现。

详细问题归类、非破坏迁移计划和验收矩阵见：docs/stage_requirements_agent_refactor_plan_2026-05-26.md。