435 lines
15 KiB
Markdown
Executable File
435 lines
15 KiB
Markdown
Executable File
# PPI ID Code Convergence Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Preserve the working PPI ID-resolution behavior while archiving speculative work, shrinking the active test surface, and simplifying only code whose behavior can be proven equivalent.
|
|
|
|
**Architecture:** Keep the existing production flow `pipeline_backend → skills/ppi/run.py → local ID map → STRING mapping → STRING network`. Archive first, repair the interrupted checkpoint, then refactor the local alias index and resolver behind their existing public interfaces. Freeze the working STRING protocol client and backend/Reset/frontend scope during this convergence.
|
|
|
|
**Tech Stack:** Python 3.11, pytest, standard-library CSV/dataclasses, requests, existing bbav5 manifests and reference bundles.
|
|
|
|
## Global Constraints
|
|
|
|
- Apply `karpathy-guidelines`: every changed active line must trace to the approved PPI ID behavior.
|
|
- Do not add Reset, CLI, Manifest, frontend, or STRING protocol features.
|
|
- Do not permanently delete current implementation or tests; archive exact pre-convergence copies first.
|
|
- Do not keep hundreds of commented lines in active modules.
|
|
- Preserve canonical IDs, exact Ortholog relationships, GeneID fallback, STRING ID/preferredName output, and empty-network versus API-failure semantics.
|
|
- Preserve working timeout, request pacing, and error classification; do not redesign the STRING client in this plan.
|
|
- Use `/home/xia-lab/miniconda3/envs/test4py311/bin/python -m pytest`; plain `pytest` is unavailable.
|
|
- The workspace is not a functional Git repository. Replace commit steps with file snapshots, test evidence, and review checkpoints.
|
|
- No live-network claim is valid unless a real STRING request and real artifact inspection are recorded.
|
|
|
|
---
|
|
|
|
### Task 1: Archive and restore a trustworthy checkpoint
|
|
|
|
**Files:**
|
|
|
|
- Create: `项目记录/代码归档/2026-07-10_PPI_ID修订收敛前快照/`
|
|
- Modify: `skills/ppi/run.py:1106-1113`
|
|
- Modify: `tests/unit/test_ppi_id_resolution.py:1421-1497`
|
|
- Create: `项目记录/代码归档/2026-07-10_PPI_ID修订收敛前快照/baseline.md`
|
|
|
|
**Interfaces:**
|
|
|
|
- Consumes: current interrupted workspace state.
|
|
- Produces: an exact recoverable snapshot and a green pre-refactor behavior checkpoint.
|
|
|
|
- [ ] **Step 1: Copy exact current files before any edit**
|
|
|
|
Archive these paths with their repository-relative layout:
|
|
|
|
```text
|
|
skills/ppi/run.py
|
|
gfa_core/id_resolution.py
|
|
gfa_core/string_rate_gate.py
|
|
gfa_core/ppi_string_policy.py
|
|
apps/web/pipeline_backend.py
|
|
tests/unit/test_id_resolution.py
|
|
tests/unit/test_ppi_id_resolution.py
|
|
tests/unit/test_ppi_string_policy.py
|
|
tests/unit/test_pipeline_backend_execution_env.py
|
|
```
|
|
|
|
Use `mkdir -p` and `cp`; do not regenerate their contents.
|
|
|
|
- [ ] **Step 2: Record the interrupted baseline**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
/home/xia-lab/miniconda3/envs/test4py311/bin/python -m pytest \
|
|
tests/unit/test_id_resolution.py \
|
|
tests/unit/test_ppi_id_resolution.py \
|
|
tests/unit/test_ppi_string_policy.py \
|
|
tests/unit/test_pipeline_backend_execution_env.py -q
|
|
```
|
|
|
|
Expected current evidence: `169 passed, 12 failed`. Record the exact output and current `wc -l` values in `baseline.md`.
|
|
|
|
- [ ] **Step 3: Repair only the interrupted `analyze_network` call**
|
|
|
|
Replace the obsolete extra positional argument with the current required signature:
|
|
|
|
```python
|
|
result = self.analyze_network(
|
|
species,
|
|
required_score,
|
|
origins=origins,
|
|
id_map_file=id_map_file,
|
|
canonical_target_ids=gene_ids,
|
|
)
|
|
```
|
|
|
|
Do not change `analyze_network`, resolution logic, or manifests in this step.
|
|
|
|
- [ ] **Step 4: Archive the four unfinished image-hardening tests**
|
|
|
|
Move the exact test bodies for these names into
|
|
`项目记录/代码归档/2026-07-10_PPI_ID修订收敛前快照/unimplemented_image_protocol_tests.py.txt` and remove them from active pytest collection:
|
|
|
|
```text
|
|
test_optional_image_rejects_deceptive_source_authority_without_http
|
|
test_optional_image_rejects_mime_or_png_signature_mismatch
|
|
test_optional_image_never_writes_svg_payload_to_png_target
|
|
```
|
|
|
|
The parametrized MIME test accounts for two collected cases, so this archives four failing cases. These tests describe unimplemented protocol expansion and are outside the approved ID scope.
|
|
|
|
- [ ] **Step 5: Verify the restored checkpoint**
|
|
|
|
Run the four-file command from Step 2.
|
|
|
|
Expected: all remaining collected tests pass. Also run:
|
|
|
|
```bash
|
|
/home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile \
|
|
gfa_core/id_resolution.py gfa_core/string_rate_gate.py \
|
|
gfa_core/ppi_string_policy.py skills/ppi/run.py apps/web/pipeline_backend.py
|
|
```
|
|
|
|
Expected: exit 0.
|
|
|
|
---
|
|
|
|
### Task 2: Replace the oversized PPI test module with a behavioral core
|
|
|
|
**Files:**
|
|
|
|
- Archive: current `tests/unit/test_ppi_id_resolution.py`
|
|
- Modify: `tests/unit/test_ppi_id_resolution.py`
|
|
- Preserve: `tests/unit/test_ppi_string_policy.py`
|
|
- Preserve: unrelated tests in `tests/unit/test_pipeline_backend_execution_env.py`
|
|
|
|
**Interfaces:**
|
|
|
|
- Consumes: public `StringAPI`, `PpiSkill.run`, `parse_ortholog_mapping`, and `StringRateGate` behavior.
|
|
- Produces: a focused test module that permits internal simplification without weakening user-visible guarantees.
|
|
|
|
- [ ] **Step 1: Archive the green full test module**
|
|
|
|
Copy the restored green file to:
|
|
|
|
`项目记录/代码归档/2026-07-10_PPI_ID修订收敛前快照/test_ppi_id_resolution.full.py.txt`
|
|
|
|
- [ ] **Step 2: Retain only these behavior tests in active collection**
|
|
|
|
Keep the fixtures needed by, and the exact test bodies for, these cases:
|
|
|
|
```text
|
|
test_parse_ortholog_mapping_preserves_every_exact_relationship
|
|
test_raw_xp_failure_falls_back_to_ncbi_geneid_and_selects_string_id
|
|
test_raw_and_geneid_mapping_to_same_string_id_is_not_ambiguous
|
|
test_one_input_mapping_to_two_distinct_string_ids_is_ambiguous
|
|
test_partial_resolution_continues_with_explicit_unresolved_rows
|
|
test_zero_resolved_ids_returns_id_resolution_failure
|
|
test_network_empty_list_is_success_empty_network
|
|
test_network_http_error_is_external_api_failure
|
|
test_get_string_ids_uses_query_index_not_response_position
|
|
test_24_origins_collapse_to_12_queries_without_losing_provenance
|
|
test_run_resolves_relationships_submits_selected_string_ids_and_writes_provenance
|
|
test_run_direct_target_raw_candidate_uses_canonical_member_evidence
|
|
test_run_counts_canonical_member_missing_ortholog_as_unresolved
|
|
test_many_canonical_targets_sharing_one_reference_keep_separate_rows
|
|
test_one_canonical_source_with_conflicting_references_is_excluded
|
|
test_mapping_404_and_200_empty_both_mean_zero_resolution
|
|
test_two_reference_queries_converging_on_one_string_id_are_both_resolved
|
|
test_request_throttle_uses_injected_clock_and_sleeper
|
|
test_interprocess_rate_gate_serializes_concurrent_clients
|
|
```
|
|
|
|
Do not rewrite assertions merely to make later refactors pass. Remove fixtures/imports only when the retained tests do not use them.
|
|
|
|
- [ ] **Step 3: Verify the behavioral core**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
/home/xia-lab/miniconda3/envs/test4py311/bin/python -m pytest \
|
|
tests/unit/test_ppi_id_resolution.py \
|
|
tests/unit/test_id_resolution.py \
|
|
tests/unit/test_ppi_string_policy.py -q
|
|
```
|
|
|
|
Expected: all collected tests pass with no network access.
|
|
|
|
- [ ] **Step 4: Check test-size reduction**
|
|
|
|
Record `wc -l` for the archived and active PPI ID test modules. The line count is evidence, not a reason to remove a distinct behavior.
|
|
|
|
---
|
|
|
|
### Task 3: Simplify the shared local alias index behind the same API
|
|
|
|
**Files:**
|
|
|
|
- Modify: `gfa_core/id_resolution.py`
|
|
- Modify only if behavior assertions need consolidation: `tests/unit/test_id_resolution.py`
|
|
|
|
**Interfaces:**
|
|
|
|
- Preserve exactly: `ReferenceAliasIndex.from_tsv(path: Path, species: str) -> ReferenceAliasIndex`
|
|
- Preserve exactly: `ReferenceAliasIndex.candidates(raw_id: str) -> list[IdAliasCandidate]`
|
|
- Preserve fields: `IdAliasCandidate(value, namespace, method, evidence_source)` and structured `IdResolutionError.code/details`.
|
|
|
|
- [ ] **Step 1: Freeze the six core alias behaviors**
|
|
|
|
First copy the green `tests/unit/test_id_resolution.py` to
|
|
`项目记录/代码归档/2026-07-10_PPI_ID修订收敛前快照/test_id_resolution.full.py.txt`.
|
|
|
|
Ensure active tests directly cover:
|
|
|
|
```text
|
|
legacy TSV exact/versionless lookup and candidate order
|
|
conflicting versioned GeneIDs are ambiguous
|
|
exact unversioned rows do not bypass version conflicts
|
|
duplicate rows with one entity deduplicate
|
|
explicit ncbi_gene_id is consumed without replacing LOC gene_id
|
|
species mismatch fails closed
|
|
```
|
|
|
|
Archive the remaining diagnostic-only cases with the full pre-convergence test snapshot; do not add new edge cases.
|
|
|
|
- [ ] **Step 2: Replace multiple evidence layers with one grouped record**
|
|
|
|
The simplified implementation may use one private frozen record per accession group:
|
|
|
|
```python
|
|
@dataclass(frozen=True, slots=True)
|
|
class _AliasRecord:
|
|
accession: str
|
|
gene_id: str
|
|
gene_symbol: str
|
|
source: str
|
|
```
|
|
|
|
Build only two indexes:
|
|
|
|
```python
|
|
self._exact: dict[str, tuple[_AliasRecord, ...]]
|
|
self._versionless: dict[str, tuple[_AliasRecord, ...]]
|
|
```
|
|
|
|
Candidate production remains ordered `accession → numeric GeneID → gene symbol`. A versionless group with more than one biological entity raises `IdResolutionError(code="ambiguous_versionless", ...)`.
|
|
|
|
- [ ] **Step 3: Run alias tests after each internal replacement**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
/home/xia-lab/miniconda3/envs/test4py311/bin/python -m pytest tests/unit/test_id_resolution.py -q
|
|
```
|
|
|
|
Expected: all retained alias behaviors pass after every small edit.
|
|
|
|
- [ ] **Step 4: Verify the PPI consumer**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
/home/xia-lab/miniconda3/envs/test4py311/bin/python -m pytest \
|
|
tests/unit/test_ppi_id_resolution.py -k 'geneid or resolution or origins' -q
|
|
```
|
|
|
|
Expected: all selected tests pass.
|
|
|
|
- [ ] **Step 5: Stop if simplification is not materially smaller**
|
|
|
|
Compare the active file with its archive. If the rewrite is not clearly shorter and easier to trace, restore the archived working file instead of keeping a lateral rewrite.
|
|
|
|
---
|
|
|
|
### Task 4: Simplify only the deterministic resolution section in `skills/ppi/run.py`
|
|
|
|
**Files:**
|
|
|
|
- Modify: `skills/ppi/run.py:436-921`
|
|
- Do not modify: `skills/ppi/run.py:51-410` (`StringAPI` and URL generation)
|
|
- Do not modify: `gfa_core/string_rate_gate.py`
|
|
- Do not modify: Reset/frontend/manifest contracts
|
|
|
|
**Interfaces:**
|
|
|
|
- Preserve `PpiQueryOrigin` and the final per-relationship resolution row fields.
|
|
- Preserve `PpiSkill.resolve_query_ids(...) -> StringResolutionResult`.
|
|
- Preserve `PpiSkill.run(...)` outputs and manifest metrics used by current tests.
|
|
|
|
- [ ] **Step 1: Map one-caller helpers before editing**
|
|
|
|
Use CodeGraph plus `rg` to record callers for:
|
|
|
|
```text
|
|
_query_candidates
|
|
_resolution_row
|
|
_exclude_conflicting_orthologs
|
|
_resolution_metrics
|
|
write_string_resolution_report
|
|
```
|
|
|
|
Only a helper with one caller and no independent behavior may be folded into that caller.
|
|
|
|
- [ ] **Step 2: Collapse candidate state to plain immutable tuples**
|
|
|
|
Replace `_ResolutionCandidate` only if all retained tests remain expressible through the final row. Use the minimal internal shape:
|
|
|
|
```python
|
|
Candidate = tuple[str, str, str, str]
|
|
# value, namespace, method, evidence_source
|
|
```
|
|
|
|
Do not change candidate order, convergence rules, or output fields.
|
|
|
|
- [ ] **Step 3: Merge row construction and conflict propagation only where single-use**
|
|
|
|
Keep one loop over `PpiQueryOrigin` that:
|
|
|
|
```text
|
|
builds ordered local candidates
|
|
looks up batched STRING results
|
|
marks resolved/unresolved/ambiguous rows
|
|
preserves every canonical-to-reference relationship
|
|
deduplicates only selected network STRING IDs
|
|
```
|
|
|
|
Do not merge STRING HTTP code into this loop.
|
|
|
|
- [ ] **Step 4: Keep metrics as a direct summary of final rows**
|
|
|
|
Derive member/query counts from final rows and the explicit canonical-member list. Preserve the real-shape assertions:
|
|
|
|
```text
|
|
24 canonical members
|
|
24 relationship rows
|
|
12 unique reference queries
|
|
5 raw resolutions
|
|
7 GeneID-only fallback resolutions
|
|
12 selected STRING queries
|
|
```
|
|
|
|
- [ ] **Step 5: Run the focused suite after each fold**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
/home/xia-lab/miniconda3/envs/test4py311/bin/python -m pytest \
|
|
tests/unit/test_ppi_id_resolution.py \
|
|
tests/unit/test_ppi_string_policy.py -q
|
|
```
|
|
|
|
Expected: all tests pass.
|
|
|
|
- [ ] **Step 6: Run backend and static regression**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
/home/xia-lab/miniconda3/envs/test4py311/bin/python -m pytest \
|
|
tests/unit/test_pipeline_backend_execution_env.py \
|
|
-k 'reference_bundle or ppi or string_network_image' -q
|
|
|
|
/home/xia-lab/miniconda3/envs/test4py311/bin/python -m py_compile \
|
|
gfa_core/id_resolution.py gfa_core/string_rate_gate.py \
|
|
gfa_core/ppi_string_policy.py skills/ppi/run.py apps/web/pipeline_backend.py
|
|
```
|
|
|
|
Expected: all selected tests pass; compilation exits 0.
|
|
|
|
- [ ] **Step 7: Apply the Karpathy stop rule**
|
|
|
|
If a proposed fold adds configuration, abstraction, compatibility dispatch, or more active lines than it removes, reject that fold and retain the archived working implementation.
|
|
|
|
---
|
|
|
|
### Task 5: Validate with real Ft→soe evidence and report convergence
|
|
|
|
**Files:**
|
|
|
|
- Create: `项目记录/2026-07-10_PPI_ID代码收敛与真实数据验证记录.md`
|
|
- Do not modify production code during this task unless a real-data failure reproduces a core acceptance criterion.
|
|
|
|
**Interfaces:**
|
|
|
|
- Consumes: current Ft canonical members, exact Ft→soe Ortholog mapping, and `soe.pep_cds_idmap.tsv` if present.
|
|
- Produces: real ID-resolution TSV/network artifacts or an exact external-network blocker report.
|
|
|
|
- [ ] **Step 1: Locate real job inputs without guessing**
|
|
|
|
Use existing job/workspace manifests and file paths. Record:
|
|
|
|
```text
|
|
canonical member file
|
|
Ortholog mapping file
|
|
soe PEP/CDS ID map
|
|
selected species/taxon
|
|
output directory
|
|
```
|
|
|
|
If an input is absent, stop the real execution and report the missing path; do not fabricate fixtures.
|
|
|
|
- [ ] **Step 2: Run the PPI skill with the real inputs**
|
|
|
|
Use the standard `test4py311` interpreter and current supported CLI/API wiring. Do not add a new CLI option during this convergence task. If the current CLI cannot pass the existing internal ID-map input, validate the real local resolver and selected STRING IDs directly and record that end-to-end wiring remains outside this plan.
|
|
|
|
- [ ] **Step 3: Inspect real artifacts**
|
|
|
|
Verify and record:
|
|
|
|
```text
|
|
canonical member count
|
|
relationship row count
|
|
unique reference query count
|
|
raw-resolved count
|
|
GeneID-fallback count
|
|
selected STRING IDs
|
|
preferredName values
|
|
network status and interaction count
|
|
```
|
|
|
|
If STRING returns `[]` normally, verify success plus `empty_network`. If HTTP/JSON fails, verify failure rather than a false zero network.
|
|
|
|
- [ ] **Step 4: Run the final regression**
|
|
|
|
Run the focused alias/PPI/policy/backend commands from Tasks 3 and 4.
|
|
|
|
Expected: all retained tests pass.
|
|
|
|
- [ ] **Step 5: Write the convergence report**
|
|
|
|
The report must include:
|
|
|
|
```text
|
|
archive path
|
|
files changed
|
|
production and test line counts before/after
|
|
tests archived and why
|
|
exact test commands/results
|
|
real Ft→soe inputs and results
|
|
live STRING validation status
|
|
remaining risks
|
|
service/Celery restart status
|
|
diagnosis record status
|
|
```
|
|
|
|
Do not claim full online validation when only local ID mapping was verified.
|