127 lines
4.2 KiB
Markdown
127 lines
4.2 KiB
Markdown
|
# Gene Family MAS Project Progress
|
||
|
|
|
||
|
|
## Backup Location
|
||
|
|
- `/home/xia-lab/gene-family-mas-backup-20260221-053220`
|
||
|
|
|
||
|
|
## Papers Location
|
||
|
|
- `/home/xia-lab/wutx/00test/bbav2/paper/gfpaper/buchong/` (33+ PDFs)
|
||
|
|
|
||
|
|
## Completed Modules (2026-02-21)
|
||
|
|
|
||
|
|
### 1. LLM Multi-Provider System (`src/llm/`)
|
||
|
|
- `base.py`: BaseLLMProvider, LLMResponse, LLMMessage
|
||
|
|
- `manager.py`: LLMManager with auto-fallback (GLM-4 → Kimi → MiniMax → DeepSeek)
|
||
|
|
- `providers/glm.py`: GLM-4 via zhipuai
|
||
|
|
- `providers/kimi.py`: Kimi via OpenAI-compatible API
|
||
|
|
- `providers/minimax.py`: MiniMax via OpenAI-compatible API
|
||
|
|
- `providers/deepseek.py`: DeepSeek via OpenAI-compatible API
|
||
|
|
|
||
|
|
### 2. Memory System (`src/memory/`)
|
||
|
|
- `base.py`: BaseMemory, MemoryEntry, SearchResult
|
||
|
|
- `session.py`: SessionMemory (in-memory with TTL)
|
||
|
|
- `case.py`: CaseMemory (SQLite persistence)
|
||
|
|
- `knowledge.py`: KnowledgeMemory (vector embeddings)
|
||
|
|
- `manager.py`: MemoryManager orchestrating all layers
|
||
|
|
|
||
|
|
### 3. Core Orchestrator (`src/core/`)
|
||
|
|
- `state.py`: GraphState (TypedDict for LangGraph)
|
||
|
|
- `orchestrator.py`: LangGraph flow with intent routing (FAQ/Analysis modes)
|
||
|
|
|
||
|
|
### 4. RAG Knowledge System (`src/rag/`) ✅
|
||
|
|
- `base.py`: Document, Chunk, SearchResult dataclasses
|
||
|
|
- `pdf_extractor.py`: PDF text extraction using PyMuPDF
|
||
|
|
- `chunker.py`: 50-30 chunking strategy
|
||
|
|
- `embeddings.py`: GLMEmbedding with fake embedding fallback
|
||
|
|
- `vector_store.py`: In-memory vector store with cosine similarity
|
||
|
|
- `retriever.py`: RAGRetriever orchestrating extraction → chunking → embedding → search
|
||
|
|
|
||
|
|
### 5. Tool Execution System (`src/tools/`) ✅
|
||
|
|
- `registry.py`: ToolRegistry with HMMER, BLAST, MAFFT, IQ-TREE, MCScanX, MEME
|
||
|
|
- `adapter.py`: ToolAdapter for command execution
|
||
|
|
- `executor.py`: ToolExecutor with Docker support
|
||
|
|
- `workflow.py`: GeneFamilyWorkflow for chaining tools
|
||
|
|
|
||
|
|
### 6. FastAPI Backend (`api/`)
|
||
|
|
- `main.py`: FastAPI app with CORS, dotenv loading
|
||
|
|
- `routes.py`: Full REST API endpoints
|
||
|
|
- `/session`, `/query`, `/case`, `/feedback`
|
||
|
|
- `/providers` - List available LLM providers
|
||
|
|
- `/tools` - List available analysis tools
|
||
|
|
- `/tools/{tool_id}/execute` - Execute a specific tool
|
||
|
|
- `/workflow/gene-family` - Run gene family workflow
|
||
|
|
- `/upload` - Single file upload
|
||
|
|
- `/upload/batch` - Batch file upload
|
||
|
|
- `/ws/{session_id}` - WebSocket for real-time progress
|
||
|
|
- `schemas.py`: Pydantic request/response models
|
||
|
|
- `websocket.py`: WebSocket connection manager
|
||
|
|
|
||
|
|
### 7. Frontend (`frontend/`)
|
||
|
|
- Next.js 14 + TypeScript + Tailwind CSS
|
||
|
|
- Chat interface with message history
|
||
|
|
- File upload support
|
||
|
|
- WebSocket for real-time progress
|
||
|
|
|
||
|
|
## API Endpoints Summary
|
||
|
|
|
||
|
|
| Endpoint | Method | Description |
|
||
|
|
|----------|--------|-------------|
|
||
|
|
| `/health` | GET | Health check |
|
||
|
|
| `/api/v1/providers` | GET | List LLM providers |
|
||
|
|
| `/api/v1/session` | POST | Create session |
|
||
|
|
| `/api/v1/query` | POST | Submit query (FAQ/Analysis) |
|
||
|
|
| `/api/v1/case` | POST/GET | Create/get case |
|
||
|
|
| `/api/v1/feedback` | POST | Submit feedback |
|
||
|
|
| `/api/v1/tools` | GET | List available tools |
|
||
|
|
| `/api/v1/tools/{tool_id}/execute` | POST | Execute tool |
|
||
|
|
| `/api/v1/workflow/gene-family` | POST | Run analysis workflow |
|
||
|
|
| `/api/v1/upload` | POST | Upload single file |
|
||
|
|
| `/api/v1/upload/batch` | POST | Upload multiple files |
|
||
|
|
| `/api/v1/ws/{session_id}` | WS | WebSocket connection |
|
||
|
|
|
||
|
|
## Configuration
|
||
|
|
|
||
|
|
### Environment Variables (.env)
|
||
|
|
```
|
||
|
|
ZHIPUAI_API_KEY=configured
|
||
|
|
MOONSHOT_API_KEY=configured
|
||
|
|
MINIMAX_API_KEY=configured
|
||
|
|
DEEPSEEK_API_KEY=configured
|
||
|
|
```
|
||
|
|
|
||
|
|
### LLM Config (config/llm.yaml)
|
||
|
|
- Default: GLM-4-flash
|
||
|
|
- Models: glm-4-flash, glm-4, glm-4-plus
|
||
|
|
- Fallback: Kimi → MiniMax → DeepSeek
|
||
|
|
|
||
|
|
## Running Commands
|
||
|
|
|
||
|
|
### Backend
|
||
|
|
```bash
|
||
|
|
cd /home/xia-lab/gene-family-mas
|
||
|
|
poetry shell
|
||
|
|
uvicorn api.main:app --host 0.0.0.0 --port 8000
|
||
|
|
```
|
||
|
|
|
||
|
|
### Frontend
|
||
|
|
```bash
|
||
|
|
cd /home/xia-lab/gene-family-mas/frontend
|
||
|
|
npm run dev
|
||
|
|
```
|
||
|
|
|
||
|
|
### Test Queries
|
||
|
|
```bash
|
||
|
|
# FAQ query
|
||
|
|
curl -X POST http://localhost:8000/api/v1/query \
|
||
|
|
-H "Content-Type: application/json" \
|
||
|
|
-d '{"session_id": "test", "query": "什么是基因家族？"}'
|
||
|
|
|
||
|
|
# Analysis query
|
||
|
|
curl -X POST http://localhost:8000/api/v1/query \
|
||
|
|
-H "Content-Type: application/json" \
|
||
|
|
-d '{"session_id": "test", "query": "帮我鉴定苦荞的SPL基因家族"}'
|
||
|
|
|
||
|
|
# File upload
|
||
|
|
curl -X POST http://localhost:8000/api/v1/upload \
|
||
|
|
-F "file=@sequences.fasta"
|
||
|
|
```
|