415 lines
12 KiB
Markdown
Executable File
415 lines
12 KiB
Markdown
Executable File
# V14 详细问答
|
|
|
|
**日期**: 2026-04-18 20:11
|
|
**版本**: V14.0
|
|
|
|
---
|
|
|
|
## 问题1: 数据层是否已完全加载到项目中？
|
|
|
|
### 答案: ✅ 是，已完全加载
|
|
|
|
**PC数据文件加载状态**:
|
|
|
|
| 数据文件 | 路径 | 数据量 | 加载方式 |
|
|
|:---|:---|:---:|:---|
|
|
| `scientific_chunks.json` | `/home/xia-lab/wutx/01model/PlantConnectome_2025version/pc/` | 43,561条别名 | `pc_data_loader.py` |
|
|
| `gene_alias.json` | 同上 | 17,618个基因 | `pc_data_loader.py` |
|
|
| `relationships_by_category_final.json` | 同上 | 788种模式 | `pc_data_loader.py` |
|
|
| `all_dic.json` | 同上 | 238万实体 | `pc_data_loader.py` |
|
|
|
|
**代码实现** (`pc_data_loader.py`):
|
|
|
|
```python
|
|
class PCDataLoader:
|
|
def load_scientific_chunks(self) -> Dict[str, str]:
|
|
"""加载43,561条别名映射"""
|
|
# 小写别名 → 标准化名称
|
|
# 示例: "aan" → "AAN"
|
|
|
|
def load_gene_aliases(self) -> Dict[str, List[str]]:
|
|
"""加载17,618个基因别名"""
|
|
# 基因 → 别名列表
|
|
# 示例: "SUC5" → ["ATSUC5", "SUC5", "AT1G71890"]
|
|
|
|
def load_relationship_patterns(self) -> Dict:
|
|
"""加载788种关系模式"""
|
|
# 9大类关系模式
|
|
# 示例: "ACTIVATION/INDUCTION/PROMOTION/CAUSATION/RESULT"
|
|
|
|
def load_entity_dict(self) -> Set[str]:
|
|
"""加载238万实体目录"""
|
|
# 用于实体验证
|
|
```
|
|
|
|
**使用方式**:
|
|
```python
|
|
from src.rag.extractors.v14 import get_pc_data_loader
|
|
|
|
loader = get_pc_data_loader()
|
|
alias_map = loader.load_scientific_chunks() # 43,561条
|
|
gene_aliases = loader.load_gene_aliases() # 17,618个
|
|
```
|
|
|
|
---
|
|
|
|
## 问题2: 输出格式中的"定义4列"是什么意思？
|
|
|
|
### 答案: 指4个定义相关的输出字段
|
|
|
|
### 2.1 4列定义字段
|
|
|
|
| 列名 | 含义 | V14实现 | PC实现 |
|
|
|:---|:---|:---:|:---:|
|
|
| `source_extracted_definition` | 从文本提取的源实体定义 | ✅ 正则提取 | ✅ 相同 |
|
|
| `source_generated_definition` | LLM生成的源实体定义 | ⚠️ **模板生成** | ✅ **LLM生成** |
|
|
| `target_extracted_definition` | 从文本提取的目标实体定义 | ✅ 正则提取 | ✅ 相同 |
|
|
| `target_generated_definition` | LLM生成的目标实体定义 | ⚠️ **模板生成** | ✅ **LLM生成** |
|
|
|
|
### 2.2 V14模板生成示例
|
|
|
|
**代码位置**: `phase4_relation_extraction.py` 第200-220行
|
|
|
|
```python
|
|
def _generate_definitions(self, relations: List[Relation], text: str) -> List[Relation]:
|
|
"""生成实体定义 (模板方式)"""
|
|
templates = {
|
|
'gene': "A gene encoding {name}, involved in biological processes.",
|
|
'tissue': "A plant tissue where {name} is expressed.",
|
|
'metabolite': "A metabolite compound {name}.",
|
|
'process': "A biological process {name}.",
|
|
'compartment': "A subcellular compartment {name}.",
|
|
}
|
|
|
|
for rel in relations:
|
|
# source定义
|
|
src_template = templates.get(rel.source_type, "A biological entity {name}.")
|
|
rel.source_generated_definition = src_template.format(
|
|
name=rel.source_resolved or rel.source
|
|
)
|
|
# 结果: "A gene encoding SUC5, involved in biological processes."
|
|
```
|
|
|
|
### 2.3 PC的LLM生成效果
|
|
|
|
**来源**: `entity_disambiguation_system.py`
|
|
|
|
```python
|
|
def generate_entity_definition(self, entity_name: str, entity_type: str, context: str = ""):
|
|
"""使用LLM生成定义"""
|
|
# 调用GPT/kimi生成
|
|
# 示例输出:
|
|
# "SUC5 (Sucrose Transporter 5) is a gene encoding a sucrose/H+ symporter
|
|
# protein that facilitates the co-transport of sucrose and protons across
|
|
# membranes, playing a crucial role in phloem loading and sugar allocation
|
|
# in Arabidopsis thaliana."
|
|
```
|
|
|
|
### 2.4 对比示例
|
|
|
|
**实体**: "SUC5"
|
|
|
|
| 方式 | 生成定义 |
|
|
|:---|:---|
|
|
| **V14模板** | "A gene encoding SUC5, involved in biological processes." |
|
|
| **PC LLM** | "SUC5 encodes a sucrose/H+ symporter involved in phloem loading..." |
|
|
|
|
**差异**:
|
|
- V14: 通用模板，简单但准确
|
|
- PC: LLM生成，详细且有生物学意义
|
|
|
|
---
|
|
|
|
## 问题3: "完整被动转主动映射"是什么意思？
|
|
|
|
### 3.1 当前V14实现
|
|
|
|
**代码位置**: `phase4_relation_extraction.py` 第45-50行
|
|
|
|
```python
|
|
def _load_passive_to_active(self) -> Dict[str, str]:
|
|
"""被动转主动映射 (V14基础版)"""
|
|
return {
|
|
'is_expressed_in': 'expressed_in',
|
|
'is_regulated_by': 'regulated_by',
|
|
'is_localized_to': 'localizes_to',
|
|
'is_involved_in': 'involved_in',
|
|
}
|
|
```
|
|
|
|
**当前处理**: 只有4个基础映射
|
|
|
|
### 3.2 PC的完整映射
|
|
|
|
**来源**: `universal_extractor_v2.py` 第150-200行 (推测)
|
|
|
|
```python
|
|
# PC的完整被动转主动映射 (推测基于788种模式)
|
|
passive_to_active = {
|
|
# 表达相关
|
|
'is_expressed_in': 'expressed_in',
|
|
'was_expressed_in': 'expressed_in',
|
|
'were_expressed_in': 'expressed_in',
|
|
'is_highly_expressed_in': 'highly_expressed_in',
|
|
|
|
# 调控相关
|
|
'is_regulated_by': 'regulated_by',
|
|
'was_up_regulated_by': 'up_regulated_by',
|
|
'was_down_regulated_by': 'down_regulated_by',
|
|
'is_induced_by': 'induced_by',
|
|
'is_repressed_by': 'repressed_by',
|
|
|
|
# 定位相关
|
|
'is_localized_to': 'localizes_to',
|
|
'was_localized_to': 'localizes_to',
|
|
'is_present_in': 'present_in',
|
|
|
|
# 功能相关
|
|
'is_involved_in': 'involved_in',
|
|
'participates_in': 'participates_in',
|
|
'plays_a_role_in': 'plays_role_in',
|
|
|
|
# ... 可能还有50+个映射
|
|
}
|
|
```
|
|
|
|
### 3.3 举例说明
|
|
|
|
**原文**: "SUC5 was up-regulated by drought stress"
|
|
|
|
| 阶段 | 处理 | 结果 |
|
|
|:---|:---|:---|
|
|
| **提取** | 正则匹配 | relation="was_up_regulated_by" |
|
|
| **标准化** | 被动转主动 | relationship_resolved="up_regulated_by" |
|
|
| **最终** | 存储 | "up_regulated_by" |
|
|
|
|
**差异**:
|
|
- V14: 可能无法识别 "was_up_regulated_by" → 保持原样
|
|
- PC: 识别为 "up_regulated_by" → 标准化
|
|
|
|
### 3.4 实体链接优化
|
|
|
|
**优化前 (V14当前)**:
|
|
```python
|
|
# 简单字符串匹配
|
|
if entity_name.lower() in entity_map:
|
|
resolved = entity_map[entity_name.lower()]
|
|
```
|
|
|
|
**问题**:
|
|
- "SUC5" 和 "suc5" 匹配
|
|
- 但 "Sucrose transporter 5" 和 "SUC5" 不匹配
|
|
|
|
**优化后 (PC风格)**:
|
|
```python
|
|
# 使用PC的scientific_chunks.json
|
|
alias_map = {
|
|
'suc5': 'SUC5',
|
|
'sucrose transporter 5': 'SUC5',
|
|
'at1g71890': 'SUC5',
|
|
'atsuc5': 'SUC5',
|
|
}
|
|
|
|
# 模糊匹配 + 别名扩展
|
|
resolved = alias_map.get(entity_name.lower(), entity_name)
|
|
```
|
|
|
|
**效果**:
|
|
- 输入: "sucrose transporter 5"
|
|
- 输出: "SUC5" (标准化)
|
|
|
|
---
|
|
|
|
## 问题4: Neo4j知识图谱集成
|
|
|
|
### 4.1 已实现的Neo4j集成
|
|
|
|
**确认**: ✅ **已实现**
|
|
|
|
**相关文件**:
|
|
|
|
| 文件 | 路径 | 功能 |
|
|
|:---|:---|:---|
|
|
| `neo4j_store.py` | `src/rag/temporal/neo4j_store.py` | 时序知识图谱Neo4j存储 |
|
|
| `import_neo4j.py` | `import_neo4j.py` | 基础Neo4j导入 |
|
|
| `import_to_neo4j.py` | `import_to_neo4j.py` | V13结果导入Neo4j |
|
|
| `demo_neo4j_import.py` | `demo_neo4j_import.py` | Neo4j导入演示 |
|
|
| `kg.db` | `kg.db` (69KB) | SQLite知识图谱数据库 |
|
|
|
|
### 4.2 时序知识图谱实现
|
|
|
|
**代码** (`src/rag/temporal/neo4j_store.py`):
|
|
|
|
```python
|
|
class TemporalNeo4jStore:
|
|
"""时序知识图谱Neo4j存储"""
|
|
|
|
def __init__(self, config: Optional[Neo4jConfig] = None):
|
|
self.config = config or Neo4jConfig()
|
|
self._driver = GraphDatabase.driver(...)
|
|
self._init_schema()
|
|
|
|
def save_temporal_entity(self, entity: TemporalEntity):
|
|
"""保存时序实体"""
|
|
# CREATE (e:TemporalEntity {entity_id: ..., valid_time: ...})
|
|
|
|
def save_temporal_relation(self, relation: TemporalRelation):
|
|
"""保存时序关系"""
|
|
# CREATE (s)-[:RELATION {valid_time: ...}]->(t)
|
|
|
|
def query_temporal_evolution(self, entity_id: str, time_range: TimePeriod):
|
|
"""查询实体时序演化"""
|
|
# MATCH (e)-[r]->() WHERE r.valid_time STARTS >= ...
|
|
```
|
|
|
|
### 4.3 使用方法
|
|
|
|
```python
|
|
from src.rag.temporal import TemporalNeo4jStore, TemporalKnowledgeGraph
|
|
|
|
# 初始化存储
|
|
store = TemporalNeo4jStore()
|
|
|
|
# 保存时序KG
|
|
kg = TemporalKnowledgeGraph()
|
|
# ... 构建KG ...
|
|
store.save_graph(kg)
|
|
|
|
# 时序查询
|
|
evolution = store.query_temporal_evolution("SUC5", TimePeriod(2020, 2024))
|
|
```
|
|
|
|
---
|
|
|
|
## 问题5: "自学习机制"是什么意思？
|
|
|
|
### 5.1 概念解释
|
|
|
|
**自学习机制**: 系统从用户反馈中自动改进，无需人工修改代码
|
|
|
|
### 5.2 举例说明
|
|
|
|
#### 场景1: 新关系模式学习
|
|
|
|
**用户反馈**:
|
|
```
|
|
用户: "这个关系 'modulates' 没有被识别"
|
|
```
|
|
|
|
**自学习过程**:
|
|
```python
|
|
# 1. 记录反馈
|
|
feedback = {
|
|
'text': "SUC5 modulates sugar uptake",
|
|
'missed_relation': 'modulates',
|
|
'context': '...',
|
|
}
|
|
|
|
# 2. 自动生成模式
|
|
new_pattern = {
|
|
'relation': 'modulates',
|
|
'pattern': r'(\b[A-Z][a-z]+[A-Z]+\d+\b).*?modulates?\s+(\w+)',
|
|
'source_type': 'gene',
|
|
'target_type': 'process',
|
|
'learned_from': feedback['text'],
|
|
}
|
|
|
|
# 3. 添加到模式库
|
|
self.relation_patterns.append(new_pattern)
|
|
|
|
# 4. 验证 (下次提取时生效)
|
|
```
|
|
|
|
#### 场景2: 别名映射学习
|
|
|
|
**用户反馈**:
|
|
```
|
|
用户: "'Suc transporter 5' 应该映射到 'SUC5'"
|
|
```
|
|
|
|
**自学习过程**:
|
|
```python
|
|
# 1. 记录别名映射
|
|
new_alias = {
|
|
'alias': 'suc transporter 5',
|
|
'canonical': 'SUC5',
|
|
'learned_from': 'user_feedback',
|
|
'confidence': 0.95,
|
|
}
|
|
|
|
# 2. 更新别名映射表
|
|
self.alias_map['suc transporter 5'] = 'SUC5'
|
|
|
|
# 3. 持久化到文件
|
|
with open('learned_aliases.json', 'a') as f:
|
|
json.dump(new_alias, f)
|
|
```
|
|
|
|
#### 场景3: 实体类型纠正
|
|
|
|
**用户反馈**:
|
|
```
|
|
用户: "'sucrose' 被错误标记为 'gene'，应该是 'metabolite'"
|
|
```
|
|
|
|
**自学习过程**:
|
|
```python
|
|
# 1. 记录纠正
|
|
correction = {
|
|
'entity': 'sucrose',
|
|
'wrong_type': 'gene',
|
|
'correct_type': 'metabolite',
|
|
}
|
|
|
|
# 2. 更新类型规则
|
|
self.entity_type_rules['sucrose'] = 'metabolite'
|
|
|
|
# 3. 调整实体识别器
|
|
if word.lower() == 'sucrose':
|
|
entity_type = self.entity_type_rules.get('sucrose', 'gene')
|
|
```
|
|
|
|
### 5.3 自学习架构
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ 自学习机制架构 │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ 用户反馈 → 反馈分类 → 模式生成 → 验证 → 持久化 → 生效 │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ 反馈类型: │
|
|
│ - 遗漏的关系模式 │
|
|
│ - 错误的别名映射 │
|
|
│ - 错误的实体类型 │
|
|
│ - 不准确的定义 │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ 学习存储: │
|
|
│ - learned_patterns.json (新关系模式) │
|
|
│ - learned_aliases.json (新别名映射) │
|
|
│ - learned_corrections.json (类型纠正) │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### 5.4 当前V14状态
|
|
|
|
**现状**: ❌ **未实现自学习机制**
|
|
|
|
**需要添加**:
|
|
1. 反馈收集接口
|
|
2. 模式生成算法
|
|
3. 验证机制
|
|
4. 持久化存储
|
|
|
|
---
|
|
|
|
## 总结
|
|
|
|
| 问题 | 状态 | 说明 |
|
|
|:---|:---:|:---|
|
|
| 数据层加载 | ✅ 完成 | 4个PC数据文件已完全加载 |
|
|
| 定义4列 | ⚠️ 模板生成 | 使用模板而非LLM，可升级 |
|
|
| 被动转主动映射 | ⚠️ 基础版 | 仅4个映射，需扩展 |
|
|
| 实体链接优化 | ⚠️ 简单匹配 | 需使用PC别名库进行模糊匹配 |
|
|
| Neo4j集成 | ✅ 已实现 | 时序知识图谱已完整实现 |
|
|
| 自学习机制 | ❌ 未实现 | 需新增反馈收集和模式生成 |
|