517 lines
14 KiB
Markdown
Executable File
517 lines
14 KiB
Markdown
Executable File
# bbav5 Code Framework Index
|
|
|
|
Date: 2026-05-20
|
|
|
|
This document is the working code map for current and future revisions. It records the code framework that has been read or indexed and marks the files that must be re-read before per-stage edits.
|
|
|
|
## 1. Frontend
|
|
|
|
### `apps/web/vue3/index.html`
|
|
|
|
Role: single-file Vue3 frontend.
|
|
|
|
Important code areas:
|
|
|
|
- Natural language input:
|
|
- `handleNlInput`
|
|
- calls `/api/agent/chat`, `/api/agent/plan`, and RAG endpoints.
|
|
- Plan display and batch execution:
|
|
- `runPlanForIntent`
|
|
- `fallbackToSimpleMode`
|
|
- `startBackgroundBatchExecution`
|
|
- `batchExecuteDialogVisible`
|
|
- `batchExecuteParams`
|
|
- `batchExecuteCountdown`
|
|
- Stage confirmation:
|
|
- `showStageConfirmDialog`
|
|
- `executeStageWithConfirmation`
|
|
- `confirmStageExecute`
|
|
- `executeStageDirect`
|
|
- `confirmDialogVisible`
|
|
- `confirmParams`
|
|
- Gene-family scope UI:
|
|
- `isGeneFamilyScopeStage`
|
|
- `geneFamilyAutoPrefix`
|
|
- target/reference/comparative checkbox groups.
|
|
- Diagnosis UI:
|
|
- `diagnosisConfirmDialogVisible`
|
|
- `diagnosisFixData`
|
|
- `confirmDiagnosisFix`
|
|
- Current UX issue:
|
|
- Prefix is still visible in several places.
|
|
- Countdown/pause logic is not fully global.
|
|
- Batch dialog has separate species-selection logic that can diverge from single-stage confirmation.
|
|
- Diagnosis display depends on task payload and is not guaranteed after every failure.
|
|
|
|
Revision direction:
|
|
|
|
- Keep ElementPlus dialog style.
|
|
- Hide prefix for gene-family scope stages.
|
|
- Make role selection the primary user-facing control.
|
|
- Centralize countdown/pause handling.
|
|
- Centralize failure diagnosis popup handling.
|
|
|
|
## 2. Backend Web Layer
|
|
|
|
### `apps/web/vue3/main.py`
|
|
|
|
Role: FastAPI backend for frontend, job management, agent endpoints, execution endpoints, task status, config autofill, scope options, diagnosis endpoints.
|
|
|
|
Important code areas:
|
|
|
|
- Prefix and scope:
|
|
- `resolve_effective_prefix`
|
|
- `_infer_family_from_config_or_final`
|
|
- `_ensure_gene_family_scope_file`
|
|
- `/api/gene-family/scope-options`
|
|
- Execution:
|
|
- `/api/execute`
|
|
- `run_stage_sync`
|
|
- `/api/task/{task_id}`
|
|
- `/api/stage-status`
|
|
- Agent endpoints:
|
|
- `/api/agent/chat`
|
|
- `/api/agent/plan`
|
|
- `/api/agent/rag`
|
|
- `/api/agent/interpret`
|
|
- Diagnosis:
|
|
- diagnosis job-stage endpoint
|
|
- confirmed-fix endpoint
|
|
- Config/autofill:
|
|
- config draft generation
|
|
- sample draft generation
|
|
- config confirm endpoint.
|
|
|
|
Current issue:
|
|
|
|
- `/api/execute` can submit a stage without mandatory preflight diagnosis.
|
|
- `run_stage_sync` can return a failed task without calling failure diagnosis.
|
|
- `/api/task/{task_id}` can only show diagnosis if it already exists in task result.
|
|
- Gene-family scope stages are partially enumerated and not global across downstream stages.
|
|
|
|
Revision direction:
|
|
|
|
- Insert `ProjectContextIndex` and `ArtifactResolver` before execution.
|
|
- Insert mandatory diagnosis on preflight/failure/postflight.
|
|
- Use one scope-generation path for all gene-family stages.
|
|
- Make task failure payload include diagnosis result.
|
|
|
|
### `apps/web/pipeline_backend.py`
|
|
|
|
Role: low-level pipeline dispatch and many file-resolution helpers.
|
|
|
|
Important functions:
|
|
|
|
- Job and upload:
|
|
- `JobInfo`
|
|
- `new_job`
|
|
- `upload_to_job`
|
|
- `list_input_table`
|
|
- `detect_role`
|
|
- `l1_advise`
|
|
- Manifest/result:
|
|
- `normalize_manifest`
|
|
- `find_latest_manifest`
|
|
- `key_results_from_manifest`
|
|
- `list_all_results`
|
|
- `preview_text`
|
|
- Scope/file helpers:
|
|
- `resolve_scope_path`
|
|
- `resolve_scoped_protein_fasta_from_phylogeny`
|
|
- `resolve_single_target_gene_id_file`
|
|
- `first_existing_scope_file`
|
|
- `resolve_species_family_pep`
|
|
- `resolve_species_sequence_file`
|
|
- `annotate_stage_manifest_with_scope`
|
|
- `ensure_protein_fasta_without_stop_markers`
|
|
- Execution:
|
|
- `run_cmd_logged`
|
|
- `run_stage`
|
|
|
|
Current issue:
|
|
|
|
- File resolution logic is useful but spread across backend and CLI scripts.
|
|
- Some helpers should move behind or be reused by a centralized `ArtifactResolver`.
|
|
|
|
Revision direction:
|
|
|
|
- Keep existing helpers initially.
|
|
- Wrap them in `ProjectContextIndex`/`ArtifactResolver`.
|
|
- Gradually remove duplicate per-stage path guessing.
|
|
|
|
## 3. Multi-Agent Core
|
|
|
|
### `gfa_core/mas/agents/router.py`
|
|
|
|
Role: first-level route classification.
|
|
|
|
Current framework:
|
|
|
|
- Keyword-driven route to analysis/query/interpretation.
|
|
- Existing result file context can route to interpretation.
|
|
- NCBI id-list context can route to analysis.
|
|
|
|
Current issue:
|
|
|
|
- It does not output a structured task object.
|
|
- It may classify result query or utility request as analysis/FAQ incorrectly.
|
|
|
|
Revision direction:
|
|
|
|
- Add task object fields: `task_type`, `operation`, `object_ref`, `scope_hint`, `file_refs`.
|
|
- Run context-aware utility/result detection before RAG fallback.
|
|
|
|
### `gfa_core/mas/agents/intent_v2.py`
|
|
|
|
Role: intent parsing and stage selection.
|
|
|
|
Current framework:
|
|
|
|
- Uses deterministic parser results where available.
|
|
- Expands stage dependencies.
|
|
- Carries `task_kind`, `utility_tool`, and params.
|
|
|
|
Current issue:
|
|
|
|
- Stage intent is decided before full project context is built.
|
|
- Prefix/species/family inference remains fragile for multi-species use.
|
|
|
|
Revision direction:
|
|
|
|
- Consume `ProjectContextIndex`.
|
|
- Separate biological scope from output prefix.
|
|
- Return semantic intent object, not only stages.
|
|
|
|
### `gfa_core/mas/utils/intent_parser.py`
|
|
|
|
Role: deterministic natural language parser.
|
|
|
|
Current framework:
|
|
|
|
- Stage keyword maps.
|
|
- Family/species extraction.
|
|
- Scope selection parsing.
|
|
- Utility request parsing for sequence extraction and NCBI fetch.
|
|
- File path extraction supports `.txt`, `.tsv`, `.csv`, `.list`, `.ids`.
|
|
|
|
Current issue:
|
|
|
|
- Regex coverage is growing case-by-case.
|
|
- File references are parsed but not resolved against job context.
|
|
|
|
Revision direction:
|
|
|
|
- Keep regex parser as fast extractor.
|
|
- Add context-backed object resolver after extraction.
|
|
- Explicitly classify result query and file transformation.
|
|
|
|
### `gfa_core/mas/agents/planning_v2.py`
|
|
|
|
Role: plan generation and preflight validation.
|
|
|
|
Current framework:
|
|
|
|
- `PlanningAgentV2`
|
|
- `REQUIRED_INPUTS`
|
|
- `_run_preflight_validation`
|
|
- `_find_input_file`
|
|
- `_check_stage_status_preflight`
|
|
- deterministic plan construction.
|
|
|
|
Current issue:
|
|
|
|
- Required input checks are file-type based and can miss existing final artifacts.
|
|
- Does not use a machine-readable stage capability registry.
|
|
- Can reject valid jobs with "missing protein".
|
|
|
|
Revision direction:
|
|
|
|
- Replace `_find_input_file` with `ArtifactResolver`.
|
|
- Preflight should return repairable alternatives.
|
|
- Plan should include resolved artifact paths and scope decision.
|
|
|
|
### `gfa_core/mas/agents/execution.py`
|
|
|
|
Role: MAS execution agent.
|
|
|
|
Current framework:
|
|
|
|
- Iterates planned stages.
|
|
- Uses `StageToolRegistry`.
|
|
- Reads log tail on error in limited cases.
|
|
|
|
Current issue:
|
|
|
|
- Does not guarantee diagnosis on failure.
|
|
- Does not apply runtime repair or bounded retry.
|
|
|
|
Revision direction:
|
|
|
|
- Execution failure must call diagnosis.
|
|
- Executor result should carry structured log, command, stage, prefix, scope.
|
|
|
|
### `gfa_core/mas/agents/diagnosis.py`
|
|
|
|
Role: error diagnosis and runtime repair.
|
|
|
|
Current framework:
|
|
|
|
- `DiagnosisAgent`
|
|
- error pattern maps.
|
|
- stage coverage maps.
|
|
- generic audits.
|
|
- stage gate diagnosis.
|
|
- runtime repairs:
|
|
- protein stop marker clean.
|
|
- empty final GFF repair.
|
|
- genome_prepare pep/cds re-extraction.
|
|
- confirmed-fix workflow for selected fixes.
|
|
|
|
Current issue:
|
|
|
|
- Good rules are not globally invoked.
|
|
- Repair policy is not expressed as a single contract.
|
|
- Some outputs are diagnostic text rather than executor-ready actions.
|
|
|
|
Revision direction:
|
|
|
|
- Normalize output to `issue`, `repair_policy`, `repair_action`, `retry_stage`.
|
|
- Add safe repairs for manifest recovery, NCBI reference symlink, PFAM fallback, artifact re-resolution.
|
|
- Wire to all execution paths.
|
|
|
|
### `gfa_core/mas/core/langgraph_orchestrator.py`
|
|
|
|
Role: LangGraph multi-agent orchestration.
|
|
|
|
Current framework:
|
|
|
|
- Router, intent, planning, preflight, execution, result parsing, validation, discovery, diagnosis, learning nodes.
|
|
|
|
Current issue:
|
|
|
|
- Execution failure can avoid the diagnosis node.
|
|
- Actual graph behavior does not match the intended "Execution -> Diagnosis -> Execution" loop.
|
|
|
|
Revision direction:
|
|
|
|
- Make failure edge go to diagnosis.
|
|
- Make diagnosis produce either repair+retry, confirmation, or final failure.
|
|
|
|
## 4. Stage Tools, Status, and Scope
|
|
|
|
### `gfa_core/mas/tools/stage_tools.py`
|
|
|
|
Role: current stage tool registry for MAS execution.
|
|
|
|
Current framework:
|
|
|
|
- `StageTool`
|
|
- `StageToolExecutor`
|
|
- `StageToolRegistry`
|
|
- stage definitions with script/category/description/required inputs/outputs/parameters/env.
|
|
|
|
Current issue:
|
|
|
|
- Stage definitions are not rich enough for semantic resolution and repair.
|
|
|
|
Revision direction:
|
|
|
|
- Use it as base data for `StageCapabilityRegistry`.
|
|
- Add consumes/provides/scope/repairs/rerun policy.
|
|
|
|
### `gfa_core/mas/tools/stage_status.py`
|
|
|
|
Role: canonical stage status checker.
|
|
|
|
Current framework:
|
|
|
|
- `manifest_path_for`
|
|
- legacy manifest migration.
|
|
- artifact path resolution.
|
|
- `get_stage_status`
|
|
- `is_stage_completed`
|
|
- `list_completed_stages`
|
|
|
|
Current value:
|
|
|
|
- Should become a required source for `ProjectContextIndex`.
|
|
|
|
### `gfa_core/gene_family_scope.py`
|
|
|
|
Role: gene-family scope builder.
|
|
|
|
Current framework:
|
|
|
|
- `GeneFamilyScope`
|
|
- `ScopeFile`
|
|
- `read_config_workbook`
|
|
- `resolve_existing_path`
|
|
- `reference_search_roots`
|
|
- `infer_family`
|
|
- `default_prefix`
|
|
- `build_scope_id`
|
|
- `selected_species_for_mode`
|
|
- `build_gene_family_scope`
|
|
- `write_scope`
|
|
- `read_scope`
|
|
|
|
Current value:
|
|
|
|
- Already has family/target/reference/comparative concepts.
|
|
|
|
Current issue:
|
|
|
|
- Scope is not applied globally to all relevant stages.
|
|
- Prefix still leaks into user-facing flow.
|
|
|
|
Revision direction:
|
|
|
|
- Keep this as the biological scope foundation.
|
|
- Make all gene-family stages consume the same scope model.
|
|
|
|
## 5. CLI Stage Framework
|
|
|
|
Core pattern across `apps/cli/run_*.py`:
|
|
|
|
```text
|
|
argparse
|
|
-> read workspace/input/final
|
|
-> resolve upstream manifest or input files
|
|
-> prepare stage directories
|
|
-> run external tools / scripts
|
|
-> copy or write final artifacts
|
|
-> write report
|
|
-> write manifest
|
|
-> write failure manifest when supported
|
|
```
|
|
|
|
### Gene-family stages
|
|
|
|
- `run_genome_prepare.py`
|
|
- prepares genome/CDS/PEP/GFF.
|
|
- must guarantee longest transcript extraction when pep/cds missing.
|
|
- must avoid empty `.gff3` and preserve `_final.gff3`.
|
|
- `run_ncbi_fetch_sequences.py`
|
|
- utility sequence download.
|
|
- should integrate with reference family naming via symlink/copy if needed.
|
|
- `run_sequence_extract_by_id.py`
|
|
- utility extraction by IDs.
|
|
- should be routed as file transformation/utility, not pipeline analysis.
|
|
- `run_identify.py`
|
|
- identifies family members.
|
|
- supports scope file, target species, PFAM/HMM params, interproscan output.
|
|
- key outputs: final members ID/PEP, interproscan status, identify manifest.
|
|
- `run_identify_refine.py`
|
|
- member refinement.
|
|
- must be audited before any domain-filter workflow is added.
|
|
- `run_phylogeny.py`
|
|
- consumes identify output and optional scope.
|
|
- can generate scoped protein input and scoped manifest.
|
|
- `run_reference_classification.py`
|
|
- classifies target genes by reference labels/tree.
|
|
- should consume the same scope and produce reusable subfamily labels.
|
|
- `run_motif.py`
|
|
- consumes protein, IDs, tree, optional subfamily.
|
|
- should reuse scoped phylogeny input.
|
|
- `run_domain.py`
|
|
- consumes protein/tree/subfamily and PFAM database.
|
|
- already has scope-aware lookup.
|
|
- `run_structure.py`
|
|
- consumes IDs, tree, target GFF, optional subfamily.
|
|
- should use scope target species, not prefix-derived species only.
|
|
- `run_combo.py`
|
|
- combines tree/motif/domain/structure.
|
|
- should be the main reference-classification support view.
|
|
- `run_promoter.py`
|
|
- consumes genome/GFF/member IDs.
|
|
- must be made scope-aware for selected target species.
|
|
- `run_synteny.py`
|
|
- intragenomic synteny.
|
|
- must use target species genome/GFF/member IDs.
|
|
- `run_interspecies_synteny.py`
|
|
- target/reference/comparative multi-species synteny.
|
|
- already has substantial scope/config discovery code.
|
|
- `run_karyotype.py`
|
|
- chromosome distribution.
|
|
- already has scope/config discovery helpers.
|
|
- `run_ortholog.py`
|
|
- ortholog mapping.
|
|
- `run_kaks_v2.py`
|
|
- Ka/Ks analysis.
|
|
- `run_ppi.py`, `run_ppi_v2.py`
|
|
- protein interaction.
|
|
- should use target IDs or ortholog mapping depending on species.
|
|
- `run_expression.py`
|
|
- gene family expression plots.
|
|
- uses `rnaR` environment.
|
|
- `run_pep.py`
|
|
- protein physicochemical features.
|
|
- can consume scoped phylogeny protein input.
|
|
- `run_annotation.py`
|
|
- functional annotation.
|
|
- should also be available as background provider for RNA-seq enrichment.
|
|
- `run_summary.py`
|
|
- should become cross-stage evidence aggregation, not only a narrow gene-family report.
|
|
|
|
### RNA-seq stages
|
|
|
|
- `run_rnaseq_qc.py`
|
|
- `run_rnaseq_align.py`
|
|
- `run_rnaseq_quant.py`
|
|
- `run_rnaseq_de.py`
|
|
- `run_rnaseq_enrich.py`
|
|
- `run_rnaseq_enrich_v2.py`
|
|
- `run_rnaseq_sample_qc.py`
|
|
- `run_rnaseq_wgcna.py`
|
|
- `run_rnaseq_fgsea.py`
|
|
- `run_rnaseq_candidate_family.py`
|
|
|
|
Revision direction:
|
|
|
|
- RNA-seq stages should be included in the same context/resolver/diagnosis/repro framework.
|
|
- They do not use target/reference/comparative in the same way, but they still need artifact resolution and repair policies.
|
|
|
|
## 6. Contracts and Skills
|
|
|
|
Contracts:
|
|
|
|
- `gfa_core/schemas/stages/*_contract_v1.md`
|
|
- Describe inputs, outputs, manifest, failure semantics.
|
|
- Useful but not machine-readable enough for automatic planning and repair.
|
|
|
|
Skills:
|
|
|
|
- `skills/*/SKILL.md`
|
|
- `skills/*/run.py`
|
|
- Some stages have skill wrappers.
|
|
|
|
Revision direction:
|
|
|
|
- Do not let contract, skill, CLI, and stage registry drift.
|
|
- Add machine-readable capability files or generate them from a single source.
|
|
- Skills should either call the same resolver or declare their consumes/provides contract.
|
|
|
|
## 7. Required Per-Stage Audit Template
|
|
|
|
Before editing any stage, fill this template in the revision notes:
|
|
|
|
```text
|
|
Stage:
|
|
CLI:
|
|
Contract:
|
|
Skill wrapper:
|
|
Frontend params:
|
|
Backend dispatch:
|
|
Input artifacts:
|
|
Output artifacts:
|
|
Manifest keys:
|
|
Scope behavior:
|
|
Prefix behavior:
|
|
Failure manifest:
|
|
Diagnosis rules:
|
|
Repairable issues:
|
|
Real-data test command:
|
|
Real-data expected outputs:
|
|
```
|
|
|