PlantAI-rag/docs/analysis/quality_comparison_and_solution.md

质量保证机制对比与解决方案

问题1: PC如何保证提取质量？

PC的质量保证机制

1. 字段完整性 (100%)

source: 100%
source_type: 100%
relationship: 100%
target: 100%
target_type: 100%
gene: 100%
pubmedID: 100%
p_source: 100%
species: 81.7%
basis: 81.7%

2. 实体定义 (89-93%)

• Source定义: 89.1%
• Target定义: 93.3%
• GPT-4o自动生成

3. 证据溯源 (81.7%)

# 每个关系都有:
{
    "basis": "RNAseq analysis and differential expression analysis",
    "p_source": "23031218_abstract",
    "pubmedID": "23031218",
    "species": "Arabidopsis thaliana"
}

4. 验证机制

• 预处理验证: GPT-4o自动验证
• 查询时验证: GPT-4o-mini实时验证
• 人工审核: 样本人工检查

5. 标准化

• 关系类型: 788个预定义关系词
• 实体类型: 预定义类型体系
• 分类体系: 9大类关系

---

问题2: 我们的章节切割为什么没有生效？

现状分析

V12.1提取结果

{
    "sections": {},  // 空！章节没有保存
    "layer1_entity_related": [],  // 空！
    "layer2_tool_doc": [],  // 空！
    "layer3_background": [32 chunks],  // 只有背景
    "gene_centric_chunks": [40 chunks]  // 但缺少详细文本
}

Gene-centric Chunks问题

SlSUT1: 514字符 (有Expression Analysis来源)
SlSUT2: 125字符 (无来源)
SlSUT3: 125字符 (无来源)
SlSUT4: 514字符 (有Expression Analysis来源)
...

问题:
1. 大部分chunks太短 (125字符)
2. 缺少来源信息
3. 没有详细描述文本
4. 没有实验方法 (basis)

根本原因

1. V12.1的章节提取没有保存

# V12.1提取了章节，但没有写入结果文件
sections = extract_sections(pdf)  # 提取了
result['sections'] = sections  # 但没有保存或保存失败

2. 三层切分逻辑问题

# Layer 1 (实体相关) 应该包含表达信息
# 但实际只有Layer 3 (背景) 有数据
# 说明实体识别和分类有问题

3. Gene-centric生成逻辑

# 当前逻辑:
for gene in filtered_genes:
    chunk = generate_summary(gene)  # 只生成摘要
    # 没有关联章节文本
    
# 应该逻辑:
for gene in filtered_genes:
    chunk = {
        "summary": generate_summary(gene),
        "expression_text": get_expression_section(gene),
        "function_text": get_function_section(gene),
        "basis": get_experimental_methods(gene)
    }

---

解决方案

方案A: 修复V12.1章节保存 (立即)

步骤1: 检查V12.1章节提取

# 检查V12.1是否正确提取章节
with open('/tmp/v12_litchi_final_test.json', 'r') as f:
    data = json.load(f)
    
print(data.keys())
# 应该有: sections, layer1, layer2, layer3

步骤2: 重新提取章节

# 使用V12.1的章节提取功能
from extract_v12_final import extract_paper_sections

sections = extract_paper_sections(pdf_path)
print(f"提取章节: {len(sections)}")
for name, text in sections.items():
    print(f"  {name}: {len(text)} 字符")

步骤3: 按基因组织章节文本

# 将章节文本关联到基因
gene_sections = {}
for gene in filtered_genes:
    gene_sections[gene] = {
        "Expression Analysis": extract_gene_from_section(
            sections.get("Expression Analysis", ""), 
            gene
        ),
        "Functional Characterization": extract_gene_from_section(
            sections.get("Functional Characterization", ""),
            gene
        ),
        "basis": extract_experimental_methods(sections)
    }

方案B: 使用原始PDF重新提取 (推荐)

步骤1: 定位Expression Analysis章节

import re

# 从PDF提取Expression Analysis章节
def extract_expression_section(pdf_text):
    patterns = [
        r'Expression Analysis\s*(.+?)(?=\n\d+\.\s+|$)',
        r'Expression Profile\s*(.+?)(?=\n\d+\.\s+|$)',
        r'Results.*?expression\s*(.+?)(?=\n\d+\.\s+|$)',
    ]
    
    for pattern in patterns:
        match = re.search(pattern, pdf_text, re.DOTALL | re.IGNORECASE)
        if match:
            return match.group(1).strip()
    
    return ""

步骤2: 从章节提取基因关系

# 使用LLM从章节文本抽取
expression_text = extract_expression_section(pdf_text)

for gene in filtered_genes:
    relations = llm_extractor.extract(
        text=expression_text,
        gene_id=gene,
        context="Expression Analysis"
    )

步骤3: 添加basis信息

# 提取实验方法
def extract_experimental_methods(text):
    methods = []
    
    if 'RT-PCR' in text or 'qPCR' in text:
        methods.append("RT-PCR")
    if 'RNA-seq' in text or 'RNAseq' in text:
        methods.append("RNA-seq")
    if 'microarray' in text.lower():
        methods.append("Microarray")
    if 'GUS' in text:
        methods.append("GUS staining")
    
    return methods

方案C: 改进Gene-centric Chunks生成 (长期)

改进后的Chunk结构

{
    "gene_id": "SlSUT1",
    "gene_name": "SlSut01",
    "species": "Solanum lycopersicum",
    "chromosome": "2",
    "location": "38551743-38555401",
    "group": "IV",
    
    # 结构化数据
    "expression": {
        "Root": "High",
        "Stem": "High",
        "Leaf": "High"
    },
    
    # 详细文本 (新增)
    "expression_text": "SlSUT1 was highly expressed in roots, stems, and leaves...",
    "function_text": "SlSUT1 encodes a sucrose transporter that...",
    "localization_text": "Subcellular localization showed that SlSUT1...",
    
    # 实验方法 (新增)
    "basis": ["RT-PCR", "RNA-seq"],
    
    # 来源 (改进)
    "sources": [
        {
            "section": "Expression Analysis",
            "paragraph": 3,
            "text": "..."
        }
    ],
    
    # 文献溯源
    "doi": "10.3390/plants13010095",
    "pmcid": "PMC10780545"
}

---

实施计划

今天 (立即)

1. 检查V12.1是否正确提取章节
2. 从原始PDF重新提取Expression Analysis
3. 使用LLM抽取表达关系

本周

1. 修复V12.1章节保存逻辑
2. 改进Gene-centric chunks生成
3. 添加basis字段

本月

1. 实现完整的PC式质量保证
2. 添加字段完整性验证
3. 实现置信度评分

---

质量对比

维度	PlantConnectome	gene-family-mas-v2 (当前)	目标
字段完整性	100%	60%	95%
实体定义	90%	0%	80%
证据溯源	82%	20%	80%
关系类型	1,458种	5种	50种
实验方法	有	无	有
置信度	有	无	有

---

关键代码

修复章节提取

# 保存章节到结果
def save_sections_to_result(sections, result_path):
    with open(result_path, 'r') as f:
        data = json.load(f)
    
    data['sections'] = sections
    
    with open(result_path, 'w') as f:
        json.dump(data, f, indent=2)

# 使用
sections = extract_sections(pdf_path)
save_sections_to_result(sections, '/tmp/v12_result.json')

改进Gene-centric生成

def generate_gene_centric_chunks_v2(genes, sections, doi, pmcid):
    chunks = []
    
    for gene in genes:
        chunk = {
            "gene_id": gene['standardized_id'],
            "chromosome": gene.get('chromosome'),
            "group": gene.get('group'),
            
            # 从章节提取详细文本
            "expression_text": extract_gene_text(
                sections.get('Expression Analysis', ''), 
                gene['standardized_id']
            ),
            "function_text": extract_gene_text(
                sections.get('Functional Characterization', ''),
                gene['standardized_id']
            ),
            
            # 实验方法
            "basis": extract_methods(sections),
            
            # 文献溯源
            "doi": doi,
            "pmcid": pmcid
        }
        chunks.append(chunk)
    
    return chunks