feat(qa-db): QA数据库裸进程标准(deploy/qa-db五脚本+守卫去版本锁定+打包/文档/Excel同步)

- deploy/qa-db/: qdrant/neo4j systemd 裸进程安装脚本+unit模板+整库替换(replace_db.sh)
  * 数据双制品模型: <部署根>/qa-db 与 install 分开传输, 支持频繁整库替换
  * 免编译(官方静态二进制/tarball)+本地 bin/ 离线分发+国内镜像回退
  * 端口 27433/27434+19888/19885(127.0.0.1), 175 docker 容器(27333/19788)为其他项目共享不动
- backend_guard.py: 移除精确计数相等校验(数据库会整库替换/持续更新, 不得锁死版本)
  保留身份校验(release/collection/kb_version/标签/关系类型), 计数仅随 health 返回
- qa/scripts/start.sh: reranker 模型路径回退(install/models/bge-reranker-base)
- build_all.sh: qa-db 脚本+reranker 模型入包; .gitignore: qa-db/bin 离线载体
- DEPLOY_FLOW.md: §0/§5.0/§5.0b/§6.0b/§7 QA 数据库裸进程标准

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-08 17:59:53 +08:00
co-authored by Claude Fable 5
parent acf5b734fa
commit e151f675f1
10 changed files with 376 additions and 43 deletions
+3
View File
@@ -18,3 +18,6 @@ coverage/
tooling/build/
bioflow_data
bioflow_service/bioflow.log
# QA 数据库裸进程离线二进制载体(构建机从镜像/官网提取,不入 git)
deploy/qa-db/bin/
+5
View File
@@ -23,6 +23,11 @@ set -a
. "$ENV_FILE"
set +a
# reranker 模型路径回退:env 未指定时用安装包内模型(install/models/bge-reranker-base)
if [[ -z "${QA_V1_BGE_RERANKER_MODEL:-}" && -d "$AGENT_ROOT/../models/bge-reranker-base" ]]; then
export QA_V1_BGE_RERANKER_MODEL="$AGENT_ROOT/../models/bge-reranker-base"
fi
cd "$AGENT_ROOT"
setsid "$PYTHON" -m uvicorn scripts.qa_v1_1_73_server:app \
--host 127.0.0.1 --port "$PORT" \
+6 -42
View File
@@ -42,15 +42,8 @@ FORBIDDEN_RELATIONSHIP_TYPES = {
"SRT2_ASSERTION_SOURCE",
"SRT2_ASSERTION_TARGET",
}
EXPECTED_NEO4J_PAPERS = 31_207
EXPECTED_NEO4J_CANONICAL_ENTITIES = 3_820_996
EXPECTED_NEO4J_CANONICAL_ASSERTIONS = 8_246_875
EXPECTED_NEO4J_CANONICAL_ASSERTS = 8_246_875
EXPECTED_NEO4J_SUPPORTED_BY = 8_202_284
EXPECTED_NORMALIZED_ENTITIES = 6_316_654
EXPECTED_NORMALIZED_PAPERS = 96_648
EXPECTED_NORMALIZED_ASSERTS = 12_963_472
EXPECTED_NORMALIZED_SUPPORTED_BY = 13_012_368
# 2026-09-08 起不再固化精确计数(数据库会频繁整库替换/更新,见 DEPLOY_FLOW.md §5.0b):
# 身份校验只看 release/collection/kb_version/标签/关系类型;实际计数随 /health 信息返回。
class BackendIdentityError(RuntimeError):
@@ -97,11 +90,7 @@ def _validate_normalized_backends() -> Dict[str, Any]:
f"{qdrant_url}/collections/{collection}"
).get("result", {})
qdrant_points = int(qdrant_info.get("points_count") or 0)
if qdrant_points != EXPECTED_NORMALIZED_ASSERTS:
raise BackendIdentityError(
"normalized Qdrant identity mismatch: "
f"expected {EXPECTED_NORMALIZED_ASSERTS} points, got {qdrant_points}"
)
# 计数不再参与校验:数据库可整库替换/持续更新(2026-09-08 决策)
sample_payload = _qdrant_sample(qdrant_url, collection)
if sample_payload.get("kb_version") != NORMALIZED_RELEASE:
raise BackendIdentityError(
@@ -167,16 +156,7 @@ def _validate_normalized_backends() -> Dict[str, Any]:
asserts_count,
supported_by_count,
)
expected = (
EXPECTED_NORMALIZED_ENTITIES,
EXPECTED_NORMALIZED_PAPERS,
EXPECTED_NORMALIZED_ASSERTS,
EXPECTED_NORMALIZED_SUPPORTED_BY,
)
if counts != expected:
raise BackendIdentityError(
f"normalized Neo4j counts mismatch: expected={expected}, got={counts}"
)
# 计数不再参与校验:数据库可整库替换/持续更新(2026-09-08 决策)
return {
"backend_policy": "neo4j_qdrant_only",
"release_id": release_id,
@@ -217,10 +197,7 @@ def validate_backends() -> Dict[str, Any]:
"result", {}
)
qdrant_points = int(qdrant_info.get("points_count") or 0)
if qdrant_points != 8_246_777:
raise BackendIdentityError(
f"Qdrant identity mismatch: expected 8246777 points, got {qdrant_points}"
)
# 计数不再参与校验:数据库可整库替换/持续更新(2026-09-08 决策)
sample_payload = _qdrant_sample(qdrant_url, collection)
if sample_payload.get("source_system") != "V15_V16_canonical":
raise BackendIdentityError(
@@ -296,20 +273,7 @@ def validate_backends() -> Dict[str, Any]:
canonical_assertions = int(counts["canonical_assertions"])
canonical_asserts = relationship_counts.get("SRT2_CANONICAL_ASSERTS", 0)
supported_by = relationship_counts.get("SRT2_SUPPORTED_BY", 0)
expected_identity = (
papers == EXPECTED_NEO4J_PAPERS
and canonical_entities == EXPECTED_NEO4J_CANONICAL_ENTITIES
and canonical_assertions == EXPECTED_NEO4J_CANONICAL_ASSERTIONS
and canonical_asserts == EXPECTED_NEO4J_CANONICAL_ASSERTS
and supported_by == EXPECTED_NEO4J_SUPPORTED_BY
)
if not expected_identity:
raise BackendIdentityError(
"Neo4j store does not match the July 2 direct canonical identity: "
f"papers={papers}, canonical_entities={canonical_entities}, "
f"canonical_assertions={canonical_assertions}, "
f"canonical_asserts={canonical_asserts}, supported_by={supported_by}"
)
# 计数不再参与校验:数据库可整库替换/持续更新(2026-09-08 决策)
return {
"backend_policy": "neo4j_qdrant_only",
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env bash
# deploy/qa-db/install_neo4j.sh —— PlantAI QA 数据库裸进程标准:Neo4j Community
# 用法(需 root):
# sudo -S bash install_neo4j.sh [QA_DB_ROOT] [BOLT_PORT] [HTTP_PORT]
# QA_DB_ROOT 默认 <安装根>/qa-db(release 树中即 install 的上一级;仓库树中即仓库根)
# BOLT_PORT 默认 19888;HTTP_PORT 默认 19885
# (与 docker 共享实例 19788/19575 互不冲突,175 上容器为其他项目服务、不动)
# 产物:/opt/neo4j-community-5.15.0 + 软链 /opt/neo4j-plantai、systemd 单元 neo4j-plantai.service
# 数据目录:$QA_DB_ROOT/neo4j/data(conf 的 server.directories.data 指向它;整库替换见 replace_db.sh)
# 版本与 2026-08 容器实例 neo4j:5.15.0-community 对齐;官方 tarball,免编译;需 JDK17(宿主已有则用)
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
INSTALL_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
QA_DB_ROOT="${1:-${QA_DB_ROOT:-$INSTALL_ROOT/qa-db}}"
BOLT_PORT="${2:-19888}"
HTTP_PORT="${3:-19885}"
NEO4J_VERSION="${NEO4J_VERSION:-5.15.0}"
RUN_USER="${RUN_USER:-${SUDO_USER:-xzq}}"
NEO4J_HOME="/opt/neo4j-community-$NEO4J_VERSION"
NEO4J_LINK=/opt/neo4j-plantai
UNIT=/etc/systemd/system/neo4j-plantai.service
TPL="$SCRIPT_DIR/neo4j.service"
DATA="$QA_DB_ROOT/neo4j/data"
LOGS="$QA_DB_ROOT/neo4j/logs"
[[ $EUID -eq 0 ]] || { echo "需要 root:sudo -S bash $0 $*" >&2; exit 1; }
command -v curl >/dev/null || { echo "缺少 curl(apt install -y curl)" >&2; exit 1; }
command -v java >/dev/null || { echo "缺少 JDK17(apt install -y openjdk-17-jre-headless)" >&2; exit 1; }
id -u "$RUN_USER" >/dev/null 2>&1 || { echo "运行用户不存在:$RUN_USER" >&2; exit 1; }
# 1) 发行包(版本已符合则跳过;官方 tarball,免编译)
if [[ -x "$NEO4J_HOME/bin/neo4j" ]]; then
echo "[neo4j] 发行包已就绪:$NEO4J_HOME"
else
# 优先使用随包携带的本地发行包(离线/受限网络部署时从构建机 scp 到此目录)
LOCAL_TGZ="$SCRIPT_DIR/bin/neo4j-community-$NEO4J_VERSION-unix.tar.gz"
if [[ -f "$LOCAL_TGZ" ]]; then
echo "[neo4j] 使用本地发行包:$LOCAL_TGZ"
mkdir -p "$NEO4J_HOME"
tar -xzf "$LOCAL_TGZ" -C /tmp
cp -a "/tmp/neo4j-community-$NEO4J_VERSION/." "$NEO4J_HOME/"
rm -rf "/tmp/neo4j-community-$NEO4J_VERSION"
else
echo "[neo4j] 下载 neo4j-community-$NEO4J_VERSION ..."
ok=0
for u in \
"https://dist.neo4j.org/neo4j-community-$NEO4J_VERSION-unix.tar.gz" \
"https://github.com/neo4j/neo4j/releases/download/$NEO4J_VERSION/neo4j-community-$NEO4J_VERSION-unix.tar.gz" \
"https://gh-proxy.com/https://github.com/neo4j/neo4j/releases/download/$NEO4J_VERSION/neo4j-community-$NEO4J_VERSION-unix.tar.gz" \
"https://ghfast.top/https://github.com/neo4j/neo4j/releases/download/$NEO4J_VERSION/neo4j-community-$NEO4J_VERSION-unix.tar.gz"; do
if curl -fL -A "Mozilla/5.0" --connect-timeout 10 --max-time 1800 -o /tmp/neo4j.tar.gz "$u" 2>/dev/null; then
ok=1; echo "[neo4j] 下载成功:$u"; break
fi
echo "[neo4j] 下载失败,换下一源:$u"
done
if [[ $ok -eq 0 ]] && command -v docker >/dev/null 2>&1 && docker image inspect "neo4j:$NEO4J_VERSION-community" >/dev/null 2>&1; then
echo "[neo4j] 回退:从本地 docker 镜像提取发行包(只读,不动运行中的容器)"
CID=$(docker create "neo4j:$NEO4J_VERSION-community")
docker cp "$CID:/var/lib/neo4j" "$NEO4J_HOME"
docker rm "$CID" >/dev/null
ok=1
fi
[[ $ok -eq 1 ]] || { echo "所有下载源失败,请手动放置发行包到 $NEO4J_HOME 或 $LOCAL_TGZ" >&2; exit 1; }
if [[ -f /tmp/neo4j.tar.gz ]]; then
mkdir -p "$NEO4J_HOME"
tar -xzf /tmp/neo4j.tar.gz -C /tmp
cp -a "/tmp/neo4j-community-$NEO4J_VERSION/." "$NEO4J_HOME/"
rm -rf /tmp/neo4j.tar.gz "/tmp/neo4j-community-$NEO4J_VERSION"
fi
fi
fi
ln -sfn "$NEO4J_HOME" "$NEO4J_LINK"
# 2) 数据/日志目录(整库替换见 replace_db.sh;目录属主对齐运行用户)
mkdir -p "$DATA" "$LOGS"
chown -R "$RUN_USER:$RUN_USER" "$QA_DB_ROOT/neo4j"
# 3) neo4j.conf(追加覆盖项;同名键后值生效)
CONF="$NEO4J_HOME/conf/neo4j.conf"
cat >> "$CONF" <<EOF
# ==== PlantAI-rag-new 裸进程覆盖项(deploy/qa-db/install_neo4j.sh 生成)====
server.bolt.listen_address=127.0.0.1:$BOLT_PORT
server.http.listen_address=127.0.0.1:$HTTP_PORT
server.directories.data=$DATA
server.directories.logs=$LOGS
dbms.security.auth_enabled=true
server.memory.heap.initial_size=16G
server.memory.heap.max_size=32G
server.memory.pagecache.size=64G
EOF
# 4) systemd 单元(模板替换占位符)
sed -e "s|@NEO4J_HOME@|$NEO4J_HOME|g" -e "s|@RUN_USER@|$RUN_USER|g" "$TPL" > "$UNIT"
# 5) 幂等启动 + 探活(bolt 端口)
systemctl daemon-reload
systemctl enable --now neo4j-plantai.service >/dev/null 2>&1 || true
for _ in $(seq 1 90); do
if (exec 3<>"/dev/tcp/127.0.0.1/$BOLT_PORT") 2>/dev/null; then
exec 3>&- 2>/dev/null || true
echo "[neo4j] UP bolt://127.0.0.1:$BOLT_PORT (data: $DATA)"
exit 0
fi
sleep 2
done
echo "[neo4j] 启动失败:journalctl -u neo4j-plantai -n 50" >&2
exit 1
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/env bash
# deploy/qa-db/install_qdrant.sh —— PlantAI QA 数据库裸进程标准:Qdrant
# 用法(需 root):
# sudo -S bash install_qdrant.sh [QA_DB_ROOT] [HTTP_PORT] [GRPC_PORT]
# QA_DB_ROOT 默认 <安装根>/qa-db(release 树中即 install 的上一级;仓库树中即仓库根)
# HTTP_PORT 默认 27433;GRPC_PORT 默认 27434
# (与 docker 共享实例 27333/27334 互不冲突,175 上容器为其他项目服务、不动)
# 产物:/usr/local/bin/qdrant、/etc/qdrant/config.yaml、systemd 单元 qdrant-plantai.service
# 数据目录:$QA_DB_ROOT/qdrant/storage(整库替换见 replace_db.sh)
# 版本默认 v1.18.3(与 2026-08 容器实例 qdrant/qdrant:latest 对齐)
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
INSTALL_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
QA_DB_ROOT="${1:-${QA_DB_ROOT:-$INSTALL_ROOT/qa-db}}"
HTTP_PORT="${2:-27433}"
GRPC_PORT="${3:-27434}"
QDRANT_VERSION="${QDRANT_VERSION:-v1.18.3}"
RUN_USER="${RUN_USER:-${SUDO_USER:-xzq}}"
BIN=/usr/local/bin/qdrant
CONF=/etc/qdrant/config.yaml
UNIT=/etc/systemd/system/qdrant-plantai.service
STORAGE="$QA_DB_ROOT/qdrant/storage"
TPL="$SCRIPT_DIR/qdrant.service"
[[ $EUID -eq 0 ]] || { echo "需要 root:sudo -S bash $0 $*" >&2; exit 1; }
command -v curl >/dev/null || { echo "缺少 curl(apt install -y curl)" >&2; exit 1; }
id -u "$RUN_USER" >/dev/null 2>&1 || { echo "运行用户不存在:$RUN_USER" >&2; exit 1; }
# 1) 二进制(版本已符合则跳过;官方静态二进制,免编译)
if [[ -x "$BIN" ]] && "$BIN" --version 2>/dev/null | grep -q "$QDRANT_VERSION"; then
echo "[qdrant] 二进制已就绪:$("$BIN" --version 2>&1 | head -1)"
else
# 优先使用随包携带的本地二进制(离线/受限网络部署时从构建机 scp 到此目录)
LOCAL_BIN="$SCRIPT_DIR/bin/qdrant"
if [[ -x "$LOCAL_BIN" ]] && "$LOCAL_BIN" --version 2>/dev/null | grep -q "$QDRANT_VERSION"; then
echo "[qdrant] 使用本地二进制:$LOCAL_BIN"
install -m 0755 "$LOCAL_BIN" "$BIN"
else
echo "[qdrant] 下载 qdrant $QDRANT_VERSION ..."
URLS=(
"https://github.com/qdrant/qdrant/releases/download/$QDRANT_VERSION/qdrant-x86_64-unknown-linux-gnu.tar.gz"
"https://gh-proxy.com/https://github.com/qdrant/qdrant/releases/download/$QDRANT_VERSION/qdrant-x86_64-unknown-linux-gnu.tar.gz"
"https://ghfast.top/https://github.com/qdrant/qdrant/releases/download/$QDRANT_VERSION/qdrant-x86_64-unknown-linux-gnu.tar.gz"
)
ok=0
for u in "${URLS[@]}"; do
if curl -fL --connect-timeout 10 --max-time 300 -o /tmp/qdrant.tar.gz "$u" 2>/dev/null; then
ok=1; echo "[qdrant] 下载成功:$u"; break
fi
echo "[qdrant] 下载失败,换下一源:$u"
done
[[ $ok -eq 1 ]] || { echo "所有下载源失败,请手动放置二进制到 $BIN 或 $LOCAL_BIN" >&2; exit 1; }
tar -xzf /tmp/qdrant.tar.gz -C /tmp
install -m 0755 /tmp/qdrant "$BIN"
rm -f /tmp/qdrant.tar.gz /tmp/qdrant
"$BIN" --version | head -1
fi
fi
# 2) 数据目录 + 配置(config 由本脚本参数生成,不手改)
mkdir -p "$STORAGE" /etc/qdrant
cat > "$CONF" <<EOF
# PlantAI-rag-new Qdrant 裸进程配置(由 deploy/qa-db/install_qdrant.sh 生成,勿手改)
storage:
storage_path: $STORAGE
snapshots_path: $STORAGE/snapshots
service:
host: 127.0.0.1
http_port: $HTTP_PORT
grpc_port: $GRPC_PORT
enable_tls: false
telemetry_disabled: true
EOF
# 数据目录属主对齐运行用户(快照恢复时文件由 qdrant 进程自己创建)
chown -R "$RUN_USER:$RUN_USER" "$QA_DB_ROOT/qdrant"
# 3) systemd 单元(模板替换占位符)
sed -e "s|@HTTP_PORT@|$HTTP_PORT|g" -e "s|@BIN@|$BIN|g" -e "s|@CONF@|$CONF|g" -e "s|@RUN_USER@|$RUN_USER|g" -e "s|@QA_DB_ROOT@|$QA_DB_ROOT|g" "$TPL" > "$UNIT"
# 4) 幂等启动 + 探活
systemctl daemon-reload
systemctl enable --now qdrant-plantai.service >/dev/null 2>&1 || true
for _ in $(seq 1 30); do
if curl -sf --max-time 2 "http://127.0.0.1:$HTTP_PORT/collections" >/dev/null 2>&1; then
echo "[qdrant] UP http://127.0.0.1:$HTTP_PORT (storage: $STORAGE)"
exit 0
fi
sleep 2
done
echo "[qdrant] 启动失败:journalctl -u qdrant-plantai -n 50" >&2
exit 1
+15
View File
@@ -0,0 +1,15 @@
[Unit]
Description=PlantAI Neo4j bare-metal instance (QA graph DB)
After=network.target
[Service]
Type=simple
User=@RUN_USER@
WorkingDirectory=@NEO4J_HOME@
Restart=always
RestartSec=5
LimitNOFILE=60000
ExecStart=@NEO4J_HOME@/bin/neo4j console
[Install]
WantedBy=multi-user.target
+15
View File
@@ -0,0 +1,15 @@
[Unit]
Description=PlantAI Qdrant bare-metal instance (QA vector DB)
After=network.target
[Service]
Type=simple
User=@RUN_USER@
WorkingDirectory=@QA_DB_ROOT@/qdrant
Restart=always
RestartSec=5
LimitNOFILE=1048576
ExecStart=@BIN@ --config-path @CONF@
[Install]
WantedBy=multi-user.target
+74
View File
@@ -0,0 +1,74 @@
#!/usr/bin/env bash
# deploy/qa-db/replace_db.sh —— PlantAI QA 裸进程数据库整库替换(用户高频操作)
# 用法(需 root;数据目录换新后属主会重对齐,按提示执行即可):
# sudo -S bash replace_db.sh qdrant <新storage目录> # 替换 Qdrant 数据
# sudo -S bash replace_db.sh neo4j <新data目录> # 替换 Neo4j 数据
# 行为:停 unit → 旧目录 mv 为 .bak.<时间戳> → 新目录就位+属主对齐 → 启 unit → 探活
# → 通过后提示删除 .bak 与回滚命令。失败自动回滚。
# 配置默认与 install_qdrant.sh / install_neo4j.sh 一致:
# QA_DB_ROOT 默认 <安装根>/qa-db;Qdrant HTTP 27433;Neo4j bolt 19888
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
INSTALL_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
QA_DB_ROOT="${QA_DB_ROOT:-$INSTALL_ROOT/qa-db}"
HTTP_PORT="${QDRANT_HTTP_PORT:-27433}"
BOLT_PORT="${NEO4J_BOLT_PORT:-19888}"
KIND="${1:-}"
NEW_DIR="${2:-}"
[[ $EUID -eq 0 ]] || { echo "需要 root:sudo -S bash $0 $*" >&2; exit 1; }
case "$KIND" in
qdrant) UNIT=qdrant-plantai; CUR="$QA_DB_ROOT/qdrant/storage"; OWNER="${SUDO_USER:-xzq}";;
neo4j) UNIT=neo4j-plantai; CUR="$QA_DB_ROOT/neo4j/data"; OWNER="${SUDO_USER:-xzq}";;
*) echo "用法: $0 qdrant|neo4j <新目录>"; exit 2;;
esac
[[ -n "$NEW_DIR" && -d "$NEW_DIR" ]] || { echo "新目录不存在:$NEW_DIR" >&2; exit 2; }
systemctl is-active --quiet "$UNIT" || { echo "unit $UNIT 未运行,请先启动或检查"; exit 2; }
TS="$(date +%Y%m%dT%H%M%S)"
BAK="${CUR}.bak.$TS"
echo "== 整库替换 $KIND =="
echo " 旧数据:$CUR"
echo " 新数据:$NEW_DIR"
echo " 备份为:$BAK"
systemctl stop "$UNIT"
sleep 3
rollback() {
echo "[回滚] 恢复旧数据并重启"
rm -rf "$CUR"
mv "$BAK" "$CUR"
chown -R "$OWNER:$OWNER" "$(dirname "$CUR")" 2>/dev/null || true
systemctl start "$UNIT"
exit 1
}
if [[ -e "$CUR" ]]; then mv "$CUR" "$BAK"; fi
mkdir -p "$(dirname "$CUR")"
mv "$NEW_DIR" "$CUR"
chown -R "$OWNER:$OWNER" "$CUR"
systemctl start "$UNIT" || { echo "[失败] 启动失败,自动回滚"; rollback; }
probe_ok=0
for _ in $(seq 1 90); do
if [[ "$KIND" == qdrant ]] && curl -sf --max-time 2 "http://127.0.0.1:$HTTP_PORT/collections" >/dev/null 2>&1; then
probe_ok=1; break
elif [[ "$KIND" == neo4j ]] && (exec 3<>"/dev/tcp/127.0.0.1/$BOLT_PORT") 2>/dev/null; then
exec 3>&- 2>/dev/null || true; probe_ok=1; break
fi
sleep 2
done
if [[ $probe_ok -eq 0 ]]; then
echo "[失败] 新库探活未通过,自动回滚"
rollback
fi
echo "== 替换成功 =="
echo " 新库在线:$CUR"
echo " 旧库备份:$BAK"
echo " 确认稳定后删除备份:rm -rf $BAK"
echo " 如需回滚:rm -rf $CUR && mv $BAK $CUR && systemctl restart $UNIT"
+43 -1
View File
@@ -19,6 +19,7 @@
| 打包产物目录 | `tooling/build/<VERSION>/install/`(VERSION 默认读 `tooling/tooling.env`,当前 2.0.0) |
| 已就绪辅助脚本 | `deploy/ollama_restart.sh`(本地嵌入模型拉起)、`vstore/vstore_restart.sh`(路径自推导重启)已入包 |
| nginx 代理模板 | `deploy/nginx/plantai.conf`(SSE 长连接 + 大文件上传适配,随安装包发布到 `install/nginx/`) |
| QA 数据库裸进程 | **systemd 裸进程为最终部署标准**(无 docker):qdrant v1.18.3 + neo4j 5.15.0-community,全部 127.0.0.1;**端口 27433/27434 + 19888/19885**;数据在 `<部署根>/qa-db/`;安装脚本 `install/qa-db/install_qdrant.sh`、`install_neo4j.sh`(均免编译:官方静态二进制/tarball,支持本地 bin/ 离线分发);175 上 docker 容器(27333/19788)为其他项目共享服务、不动 |
## 1. 编译阶段(构建机)
@@ -108,8 +109,40 @@ rsync -av --progress tooling/build/2.0.1/install/ <target-host>:/srv/plantai/
## 5. 目标机部署
### 5.0 QA 数据库(systemd 裸进程,先于业务服务)
```bash
cd /srv/plantai/install
bash qa-db/install_qdrant.sh # 需 sudo;默认装 /usr/local/bin/qdrant + 27433/27434
bash qa-db/install_neo4j.sh # 需 sudo;默认装 /opt/neo4j-community-5.15.0 + 19888/19885
# 全新部署先按 5.0b 同步数据库文件再启动;换库用 qa-db/replace_db.sh
```
### 5.0b 数据库数据同步与整库替换(数据与安装包双制品分开传输)
> **数据库文件不进 install 安装包**(否则每次发布拖 82G 且整树覆盖会波及在用的库)。
> 全新部署/整库更新时,按本小节显式同步到目标机 `<部署根>/qa-db/`;程序发布只传 install/。
```bash
# 1) 从源机(现网/构建机)同步数据库到目标机(显式步骤,与 install 包传输并列):
rsync -av --progress <源机>:/<源部署根>/qa-db/qdrant/storage/ /srv/plantai/qa-db/qdrant/storage/
rsync -av --progress <源机>:/<源部署根>/qa-db/neo4j/data/ /srv/plantai/qa-db/neo4j/data/
# 2) 属主对齐(install 脚本也会做;rsync 后建议立即执行):
sudo chown -R <运行用户>:<运行用户> /srv/plantai/qa-db/{qdrant,neo4j}
# 3) 启动/重启裸进程(探活 curl 27433/collections 与 bolt 19888):
sudo systemctl restart qdrant-plantai neo4j-plantai
# 4) 整库替换(用户高频操作,数据目录换新,自动探活+回滚):
sudo bash qa-db/replace_db.sh qdrant <新storage目录>
sudo bash qa-db/replace_db.sh neo4j <新data目录>
```
- **位置约定**:数据根 = `<部署根>/qa-db/`(默认 /srv/plantai/qa-db;可用 QA_DB_ROOT env 覆盖,install 脚本与 replace_db.sh 都认)。
- **免编译说明**:qdrant 官方 release 静态二进制、neo4j 官方 community tarball,均无需编译;版本校验 = `qdrant --version`(v1.18.3)与 `neo4j version`(5.15.0)。受限网络用 `qa-db/bin/` 离线载体(install 包自带,无则自动回退下载含国内镜像源)。
- **运行用户**:RUN_USER 默认 xzq(sudo 装 systemd 时自动取 SUDO_USER);systemd 单元已配 WorkingDirectory 与重启策略。
### 5.1 环境准备
```bash
# 5.1 环境准备(目标机):
# - 安装 py3.12 venv(或复用构建机 venv 的 rsync 拷贝),安装依赖(sdk/pyproject.toml 与 vstore/pyproject.toml 的 dependencies)
# - 编辑 .env:sdk/.env(DOCGEN_LLM_* 与 VSTORE_BASE_URL)、vstore/.env(VSTORE_KEY、VSTORE_ROOT、VSTORE_EMBED_URL、VSTORE_EMBED_MODEL=bge-m3:567m)
# - KB 密钥(若 SKIP_KB=0):把 kb.key 放到目标机并 chmod 600
@@ -168,6 +201,14 @@ nginx 模板关键适配(勿删勿改,详见 `deploy/nginx/plantai.conf` 注
# 6.1 进程与端口
ss -tln | grep -E ':(8090|8091|8092|8093|8094|8098)' # 8090=0.0.0.0,其余 127.0.0.1
# 6.0b QA 数据库裸进程专项
systemctl is-active qdrant-plantai neo4j-plantai # 均 active
ss -tln | grep -E ':(27433|27434|19888|19885)' # 全部 127.0.0.1
curl -s http://127.0.0.1:27433/collections | python3 -m json.tool | head -8 # 含 plant_srt_pc2025_v3_20260817_normalized
curl -s http://127.0.0.1:8091/health | python3 -m json.tool | head -8 # neo4j_entities 6438960 / qdrant_points 13040841
# 经 19888 bolt 登录验证(凭据用 QA 运行中进程的 NEO4J_PASSWORD,与容器一致):
/opt/neo4j-community-5.15.0/bin/cypher-shell -a bolt://127.0.0.1:19888 -u neo4j -p "$NEO4J_PASSWORD" "MATCH (n) RETURN count(n);"
# 6.1b nginx 专项
systemctl is-active nginx 2>/dev/null || pgrep -x nginx # nginx 存活
ss -tln | grep ':80 ' # 80 端口监听(0.0.0.0 或目标机内网 IP)
@@ -212,3 +253,4 @@ curl -s 'http://127.0.0.1:8094/api/vstore/search?q=测试&as_template=false' | p
4. **构建机与目标机若为同一台机器**(内网 192.168.0.175 现状),部署就是替换现有 `plantai-rag-new-deploy-unified/` 部署树;执行前先备份、执行后按 §6 全量质检。
5. 本手册基于 2026-09-08 服务器实测编写;如工具链或目录结构有变,先对照仓库实际脚本修正本手册再执行。
6. **当前生产服务器(192.168.0.175)仍是 0.0.0.0:8090 直连形态,尚未执行 §5b 的 nginx 迁移**——启动脚本默认值已改为 127.0.0.1,但现网进程仍是旧绑定;迁移时按 §5b 顺序执行(先 nginx 后改绑),不要直接无参重启 bff_restart.sh 以免客户访问断档。
7. **175 上 QA 数据库 docker 容器(27333/19788)为其他项目共享服务,不随 PlantAI 部署流程管理**;PlantAI-rag-new 一律走裸进程(§5.0/5.0b);遗留库(26333/18788)现网无引用,清理需用户单独命令。
+15
View File
@@ -103,6 +103,21 @@ cp "$PROJECT_ROOT/deploy/ollama_restart.sh" "$RELEASE_ROOT/ollama_restart.sh"
# nginx 80 端口反代模板(目标机从零安装章节引用,见 docs/DEPLOY_FLOW.md §5b)
mkdir -p "$RELEASE_ROOT/nginx"
cp "$PROJECT_ROOT/deploy/nginx/plantai.conf" "$RELEASE_ROOT/nginx/plantai.conf"
# QA 数据库裸进程部署脚本(systemd 标准,见 docs/DEPLOY_FLOW.md §5「QA 数据库」)
mkdir -p "$RELEASE_ROOT/qa-db"
cp -r "$PROJECT_ROOT/deploy/qa-db/"*.sh "$RELEASE_ROOT/qa-db/" 2>/dev/null || true
cp -r "$PROJECT_ROOT/deploy/qa-db/"*.service "$RELEASE_ROOT/qa-db/" 2>/dev/null || true
# 离线二进制载体(deploy/qa-db/bin/ 为 gitignore;存在则随包分发,目标机免下载)
[ -d "$PROJECT_ROOT/deploy/qa-db/bin" ] && cp -r "$PROJECT_ROOT/deploy/qa-db/bin" "$RELEASE_ROOT/qa-db/bin" || true
# QA reranker 模型入包(构建机仓库同级 PlantAI-rag;缺失只告警不阻断)
RERANKER_SRC="$PROJECT_ROOT/../PlantAI-rag/bge-reranker-base"
if [ -d "$RERANKER_SRC" ]; then
mkdir -p "$RELEASE_ROOT/models"
cp -r "$RERANKER_SRC" "$RELEASE_ROOT/models/bge-reranker-base"
echo "reranker 模型已入包: models/bge-reranker-base"
else
echo "警告: reranker 模型源不存在($RERANKER_SRC),安装包不含本地 reranker"
fi
# env 模板(VSTORE_ROOT/KEY/EMBED 等;真实值不入包)
cp "$PROJECT_ROOT/vstore/.env.example" "$RELEASE_ROOT/vstore/.env.example" 2>/dev/null || true