337 lines
7.9 KiB
Markdown
Executable File
337 lines
7.9 KiB
Markdown
Executable File
# 质量保证机制对比与解决方案
|
|
|
|
## 问题1: PC如何保证提取质量？
|
|
|
|
### PC的质量保证机制
|
|
|
|
#### 1. 字段完整性 (100%)
|
|
```
|
|
source: 100%
|
|
source_type: 100%
|
|
relationship: 100%
|
|
target: 100%
|
|
target_type: 100%
|
|
gene: 100%
|
|
pubmedID: 100%
|
|
p_source: 100%
|
|
species: 81.7%
|
|
basis: 81.7%
|
|
```
|
|
|
|
#### 2. 实体定义 (89-93%)
|
|
- Source定义: 89.1%
|
|
- Target定义: 93.3%
|
|
- GPT-4o自动生成
|
|
|
|
#### 3. 证据溯源 (81.7%)
|
|
```python
|
|
# 每个关系都有:
|
|
{
|
|
"basis": "RNAseq analysis and differential expression analysis",
|
|
"p_source": "23031218_abstract",
|
|
"pubmedID": "23031218",
|
|
"species": "Arabidopsis thaliana"
|
|
}
|
|
```
|
|
|
|
#### 4. 验证机制
|
|
- **预处理验证**: GPT-4o自动验证
|
|
- **查询时验证**: GPT-4o-mini实时验证
|
|
- **人工审核**: 样本人工检查
|
|
|
|
#### 5. 标准化
|
|
- **关系类型**: 788个预定义关系词
|
|
- **实体类型**: 预定义类型体系
|
|
- **分类体系**: 9大类关系
|
|
|
|
---
|
|
|
|
## 问题2: 我们的章节切割为什么没有生效？
|
|
|
|
### 现状分析
|
|
|
|
#### V12.1提取结果
|
|
```json
|
|
{
|
|
"sections": {}, // 空！章节没有保存
|
|
"layer1_entity_related": [], // 空！
|
|
"layer2_tool_doc": [], // 空！
|
|
"layer3_background": [32 chunks], // 只有背景
|
|
"gene_centric_chunks": [40 chunks] // 但缺少详细文本
|
|
}
|
|
```
|
|
|
|
#### Gene-centric Chunks问题
|
|
```
|
|
SlSUT1: 514字符 (有Expression Analysis来源)
|
|
SlSUT2: 125字符 (无来源)
|
|
SlSUT3: 125字符 (无来源)
|
|
SlSUT4: 514字符 (有Expression Analysis来源)
|
|
...
|
|
|
|
问题:
|
|
1. 大部分chunks太短 (125字符)
|
|
2. 缺少来源信息
|
|
3. 没有详细描述文本
|
|
4. 没有实验方法 (basis)
|
|
```
|
|
|
|
### 根本原因
|
|
|
|
#### 1. V12.1的章节提取没有保存
|
|
```python
|
|
# V12.1提取了章节，但没有写入结果文件
|
|
sections = extract_sections(pdf) # 提取了
|
|
result['sections'] = sections # 但没有保存或保存失败
|
|
```
|
|
|
|
#### 2. 三层切分逻辑问题
|
|
```python
|
|
# Layer 1 (实体相关) 应该包含表达信息
|
|
# 但实际只有Layer 3 (背景) 有数据
|
|
# 说明实体识别和分类有问题
|
|
```
|
|
|
|
#### 3. Gene-centric生成逻辑
|
|
```python
|
|
# 当前逻辑:
|
|
for gene in filtered_genes:
|
|
chunk = generate_summary(gene) # 只生成摘要
|
|
# 没有关联章节文本
|
|
|
|
# 应该逻辑:
|
|
for gene in filtered_genes:
|
|
chunk = {
|
|
"summary": generate_summary(gene),
|
|
"expression_text": get_expression_section(gene),
|
|
"function_text": get_function_section(gene),
|
|
"basis": get_experimental_methods(gene)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 解决方案
|
|
|
|
### 方案A: 修复V12.1章节保存 (立即)
|
|
|
|
#### 步骤1: 检查V12.1章节提取
|
|
```python
|
|
# 检查V12.1是否正确提取章节
|
|
with open('/tmp/v12_litchi_final_test.json', 'r') as f:
|
|
data = json.load(f)
|
|
|
|
print(data.keys())
|
|
# 应该有: sections, layer1, layer2, layer3
|
|
```
|
|
|
|
#### 步骤2: 重新提取章节
|
|
```python
|
|
# 使用V12.1的章节提取功能
|
|
from extract_v12_final import extract_paper_sections
|
|
|
|
sections = extract_paper_sections(pdf_path)
|
|
print(f"提取章节: {len(sections)}")
|
|
for name, text in sections.items():
|
|
print(f" {name}: {len(text)} 字符")
|
|
```
|
|
|
|
#### 步骤3: 按基因组织章节文本
|
|
```python
|
|
# 将章节文本关联到基因
|
|
gene_sections = {}
|
|
for gene in filtered_genes:
|
|
gene_sections[gene] = {
|
|
"Expression Analysis": extract_gene_from_section(
|
|
sections.get("Expression Analysis", ""),
|
|
gene
|
|
),
|
|
"Functional Characterization": extract_gene_from_section(
|
|
sections.get("Functional Characterization", ""),
|
|
gene
|
|
),
|
|
"basis": extract_experimental_methods(sections)
|
|
}
|
|
```
|
|
|
|
### 方案B: 使用原始PDF重新提取 (推荐)
|
|
|
|
#### 步骤1: 定位Expression Analysis章节
|
|
```python
|
|
import re
|
|
|
|
# 从PDF提取Expression Analysis章节
|
|
def extract_expression_section(pdf_text):
|
|
patterns = [
|
|
r'Expression Analysis\s*(.+?)(?=\n\d+\.\s+|$)',
|
|
r'Expression Profile\s*(.+?)(?=\n\d+\.\s+|$)',
|
|
r'Results.*?expression\s*(.+?)(?=\n\d+\.\s+|$)',
|
|
]
|
|
|
|
for pattern in patterns:
|
|
match = re.search(pattern, pdf_text, re.DOTALL | re.IGNORECASE)
|
|
if match:
|
|
return match.group(1).strip()
|
|
|
|
return ""
|
|
```
|
|
|
|
#### 步骤2: 从章节提取基因关系
|
|
```python
|
|
# 使用LLM从章节文本抽取
|
|
expression_text = extract_expression_section(pdf_text)
|
|
|
|
for gene in filtered_genes:
|
|
relations = llm_extractor.extract(
|
|
text=expression_text,
|
|
gene_id=gene,
|
|
context="Expression Analysis"
|
|
)
|
|
```
|
|
|
|
#### 步骤3: 添加basis信息
|
|
```python
|
|
# 提取实验方法
|
|
def extract_experimental_methods(text):
|
|
methods = []
|
|
|
|
if 'RT-PCR' in text or 'qPCR' in text:
|
|
methods.append("RT-PCR")
|
|
if 'RNA-seq' in text or 'RNAseq' in text:
|
|
methods.append("RNA-seq")
|
|
if 'microarray' in text.lower():
|
|
methods.append("Microarray")
|
|
if 'GUS' in text:
|
|
methods.append("GUS staining")
|
|
|
|
return methods
|
|
```
|
|
|
|
### 方案C: 改进Gene-centric Chunks生成 (长期)
|
|
|
|
#### 改进后的Chunk结构
|
|
```python
|
|
{
|
|
"gene_id": "SlSUT1",
|
|
"gene_name": "SlSut01",
|
|
"species": "Solanum lycopersicum",
|
|
"chromosome": "2",
|
|
"location": "38551743-38555401",
|
|
"group": "IV",
|
|
|
|
# 结构化数据
|
|
"expression": {
|
|
"Root": "High",
|
|
"Stem": "High",
|
|
"Leaf": "High"
|
|
},
|
|
|
|
# 详细文本 (新增)
|
|
"expression_text": "SlSUT1 was highly expressed in roots, stems, and leaves...",
|
|
"function_text": "SlSUT1 encodes a sucrose transporter that...",
|
|
"localization_text": "Subcellular localization showed that SlSUT1...",
|
|
|
|
# 实验方法 (新增)
|
|
"basis": ["RT-PCR", "RNA-seq"],
|
|
|
|
# 来源 (改进)
|
|
"sources": [
|
|
{
|
|
"section": "Expression Analysis",
|
|
"paragraph": 3,
|
|
"text": "..."
|
|
}
|
|
],
|
|
|
|
# 文献溯源
|
|
"doi": "10.3390/plants13010095",
|
|
"pmcid": "PMC10780545"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 实施计划
|
|
|
|
### 今天 (立即)
|
|
1. 检查V12.1是否正确提取章节
|
|
2. 从原始PDF重新提取Expression Analysis
|
|
3. 使用LLM抽取表达关系
|
|
|
|
### 本周
|
|
1. 修复V12.1章节保存逻辑
|
|
2. 改进Gene-centric chunks生成
|
|
3. 添加basis字段
|
|
|
|
### 本月
|
|
1. 实现完整的PC式质量保证
|
|
2. 添加字段完整性验证
|
|
3. 实现置信度评分
|
|
|
|
---
|
|
|
|
## 质量对比
|
|
|
|
| 维度 | PlantConnectome | gene-family-mas-v2 (当前) | 目标 |
|
|
|------|-----------------|--------------------------|------|
|
|
| 字段完整性 | 100% | 60% | 95% |
|
|
| 实体定义 | 90% | 0% | 80% |
|
|
| 证据溯源 | 82% | 20% | 80% |
|
|
| 关系类型 | 1,458种 | 5种 | 50种 |
|
|
| 实验方法 | 有 | 无 | 有 |
|
|
| 置信度 | 有 | 无 | 有 |
|
|
|
|
---
|
|
|
|
## 关键代码
|
|
|
|
### 修复章节提取
|
|
```python
|
|
# 保存章节到结果
|
|
def save_sections_to_result(sections, result_path):
|
|
with open(result_path, 'r') as f:
|
|
data = json.load(f)
|
|
|
|
data['sections'] = sections
|
|
|
|
with open(result_path, 'w') as f:
|
|
json.dump(data, f, indent=2)
|
|
|
|
# 使用
|
|
sections = extract_sections(pdf_path)
|
|
save_sections_to_result(sections, '/tmp/v12_result.json')
|
|
```
|
|
|
|
### 改进Gene-centric生成
|
|
```python
|
|
def generate_gene_centric_chunks_v2(genes, sections, doi, pmcid):
|
|
chunks = []
|
|
|
|
for gene in genes:
|
|
chunk = {
|
|
"gene_id": gene['standardized_id'],
|
|
"chromosome": gene.get('chromosome'),
|
|
"group": gene.get('group'),
|
|
|
|
# 从章节提取详细文本
|
|
"expression_text": extract_gene_text(
|
|
sections.get('Expression Analysis', ''),
|
|
gene['standardized_id']
|
|
),
|
|
"function_text": extract_gene_text(
|
|
sections.get('Functional Characterization', ''),
|
|
gene['standardized_id']
|
|
),
|
|
|
|
# 实验方法
|
|
"basis": extract_methods(sections),
|
|
|
|
# 文献溯源
|
|
"doi": doi,
|
|
"pmcid": pmcid
|
|
}
|
|
chunks.append(chunk)
|
|
|
|
return chunks
|
|
```
|