Files
potaser 55410b663a feat(review): journal format-rule fetch scripts (offline CLI)
- fetch_format_rules.py CLI: robots.txt-aware fetcher (httpx + bs4,
  >=2s per-host throttle) + DeepSeek quote-backed rule extractor
- Status: OUP/Wiley/Cell return 403 from this environment; the five
  featured journals' format_rules in journals.json were manually
  verified against official sites instead. Tooling kept for future
  re-runs; data/format_cache/ is runtime output and gitignored
- Service code never imports this toolchain (standalone CLI only)
2026-10-11 12:19:18 +08:00

1006 B
Raw Permalink Blame History

scripts/ —— 离线工具(不被 app/ 服务代码引用,独立 CLI)

fetch_format_rules.py —— 投稿指南自动抓取(探索性,第一批)

  • 依赖:venv 已装的 httpx + beautifulsoup4 4.15.0(用户拍板 2026-10-10); 解析器 lxml 6.1.2 在 tooling/requirements.lock 内。
  • 用法与流程见 fetch_format_rules.py 模块 docstring(CLI 一次一本期刊)。
  • 产物:data/format_cache/{journal_id}.json(运行产物,已 gitignore,不入库)。

现状(2026-10-10)

OUP / Wiley / Cell 官网对本运行环境返回 403,自动抓取路线失败; Genome Biology / BMC Plant Biology / Frontiers / Rice 可抓但 LLM 抽取结果 仍需人工对照官网。当前 journals.json 中 5 本精选期刊的 format_rules 为人工核验官网原文后写入(rules_source=official + source_url + last_verified),不是本脚本产物。本工具保留作为后续换网络环境 / 换可达站点后的重跑手段。