Add doctor_all, prune, model docs, and selftest CI.
selftest / selftest (push) Canceled after 0s

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
swl
2026-10-10 02:06:54 +08:00
co-authored by Cursor
parent 1de2654858
commit dbfb9b52f9
8 changed files with 217 additions and 30 deletions
+37
View File
@@ -0,0 +1,37 @@
name: selftest
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
selftest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install
run: |
python -m pip install -U pip
pip install -r requirements.txt
- name: Stub ComfyUI layout
run: |
mkdir -p models/checkpoints models/loras output/MyMusic user/default/workflows/bak
touch models/checkpoints/ace_step_1.5_turbo_swl_fused.safetensors
touch models/loras/swl_voice_best_ep257.safetensors
touch user/default/workflows/song_rules.py
touch "user/default/workflows/urbanhiphop-身体先听懂.json"
echo stub > user/default/workflows/bak/ci-stub
- name: Layout for package import
run: |
mkdir -p /tmp/sfwrap
ln -s "$GITHUB_WORKSPACE" /tmp/sfwrap/songforge
- name: selftest
working-directory: /tmp/sfwrap
run: python songforge/songforge.py selftest
env:
SONGFORGE_COMFY_ROOT: ${{ github.workspace }}
+9 -7
View File
@@ -55,12 +55,9 @@ git clone https://git.nnrc.ren/swl/songforge.git
# 或 https://github.com/lsuen/songforge.git # 或 https://github.com/lsuen/songforge.git
cd songforge cd songforge
.\scripts\setup_venv.ps1 .\scripts\doctor_all.ps1
copy config\local.toml.example config\local.toml # 缺 local.toml 时会复制 example 并退出;编辑并启动 ComfyUI 后再跑
# 编辑 local.toml:engine.server / paths.comfy_root / qc_python / music_dir # 模型文件名见 README「需要的模型」;没有作者 LoRA 时先问用户
# 用户需先启动本机 ComfyUI
.\.venv\Scripts\python.exe songforge.py doctor
``` ```
入口(在**仓根**): 入口(在**仓根**):
@@ -156,17 +153,22 @@ doctor → target → 写歌词 → check → gen → qc --baseline
|------|------| |------|------|
| 无 local.toml | **问用户**路径后创建并填写 | | 无 local.toml | **问用户**路径后创建并填写 |
| doctor FAIL 服务 | 请用户启动 ComfyUI;核对 server | | doctor FAIL 服务 | 请用户启动 ComfyUI;核对 server |
| doctor FAIL 模型 | **问**模型是否在该 comfy_root | | doctor FAIL 模型 | **问**模型是否在该 comfy_root;对照 README 文件名 |
| check 密度失败 | 改歌词到 target 区间 | | check 密度失败 | 改歌词到 target 区间 |
| gen 提交失败 | `selftest`;必要时 `--dry-run` | | gen 提交失败 | `selftest`;必要时 `--dry-run` |
| qc voice_nn=null | 补 `--baseline` | | qc voice_nn=null | 补 `--baseline` |
| qc FAIL | 修歌/重 gen,**不**改阈值;反复失败则**问用户**听哪一版 | | qc FAIL | 修歌/重 gen,**不**改阈值;反复失败则**问用户**听哪一版 |
| tmp 太肥 | `prune --days 7`(不动 runs/音频) |
| CLI 像坏了 | `selftest`;仍异常则**问用户**并贴 log | | CLI 像坏了 | `selftest`;仍异常则**问用户**并贴 log |
| 要 push | **问**推 `gitea` 还是 GitHub;勿推 secrets/archive |
```powershell ```powershell
.\.venv\Scripts\python.exe songforge.py selftest .\.venv\Scripts\python.exe songforge.py selftest
.\.venv\Scripts\python.exe songforge.py prune --days 7 --dry-run
``` ```
冒烟歌词:`input/lyrics/pop_灯还亮着.txt`。
--- ---
## 8. 验收清单 ## 8. 验收清单
+43 -16
View File
@@ -12,6 +12,14 @@
> 给 Agent:读完 `AGENTS.md` 再动手。缺 `local.toml`、Comfy 没起、风格/歌名不清、想改模型或阈值时——**先问用户,不要猜**。 > 给 Agent:读完 `AGENTS.md` 再动手。缺 `local.toml`、Comfy 没起、风格/歌名不清、想改模型或阈值时——**先问用户,不要猜**。
## 一键体检
```powershell
.\scripts\doctor_all.ps1
```
会:缺 `.venv` 则创建 → 缺 `local.toml` 则从 example 复制并提示编辑 → 跑 `doctor` → 打印下一步。
## 换目录也能用(推荐) ## 换目录也能用(推荐)
```powershell ```powershell
@@ -20,23 +28,30 @@ git clone https://git.nnrc.ren/swl/songforge.git
# 或:git clone https://github.com/lsuen/songforge.git # 或:git clone https://github.com/lsuen/songforge.git
cd songforge cd songforge
.\scripts\setup_venv.ps1 .\scripts\doctor_all.ps1
copy config\local.toml.example config\local.toml # 按提示编辑 config\local.toml 后,启动 ComfyUI,再跑一次 doctor_all
# 编辑 local.toml:
# [engine] server = "http://127.0.0.1:8188"
# [paths] comfy_root / qc_python / music_dir
# 先启动本机 ComfyUI,再:
.\.venv\Scripts\python.exe songforge.py doctor
``` ```
不必把本仓放进 ComfyUI 的 `user/default/workflows`。 不必把本仓放进 ComfyUI 的 `user/default/workflows`。
## 需要的模型(ComfyUI 侧)
在 `local.toml` 的 `comfy_root` 下准备:
| 类型 | 文件名 | 目录 |
|------|--------|------|
| Checkpoint | `ace_step_1.5_turbo_swl_fused.safetensors` | `models/checkpoints/` |
| LoRA | `swl_voice_best_ep257.safetensors` | `models/loras/` |
- 默认 LoRA 是**作者定制声线**;别人没有同文件时 `doctor` 会 FAIL。
- 可换成自己的 LoRA,并改 `config/default.toml` 的 `[model]`(声线会变;不要用禁用名单里的 YuE2/aio)。
- 缺文件时 `doctor` 会打印人话排查步骤。
## 两个 Python 环境 ## 两个 Python 环境
| 环境 | 用途 | | 环境 | 用途 |
|------|------| |------|------|
| **本仓 `.venv`** | CLI:`doctor` / `check` / `gen` / `catalog` | | **本仓 `.venv`** | CLI:`doctor` / `check` / `gen` / `catalog` / `prune` |
| **`qc_python`** | QC 音频栈(可继续用 ComfyUI 的 venv) | | **`qc_python`** | QC 音频栈(可继续用 ComfyUI 的 venv) |
## 配置:TOML vs JSON ## 配置:TOML vs JSON
@@ -55,26 +70,38 @@ copy config\local.toml.example config\local.toml
```powershell ```powershell
.\.venv\Scripts\python.exe songforge.py target --style pop --bpm 104 --dur 180 .\.venv\Scripts\python.exe songforge.py target --style pop --bpm 104 --dur 180
.\.venv\Scripts\python.exe songforge.py check --style pop --title 歌名 --lyrics-file input\lyrics\歌名.txt .\.venv\Scripts\python.exe songforge.py check --style pop --title 灯还亮着 --lyrics-file input\lyrics\pop_灯还亮着.txt
.\.venv\Scripts\python.exe songforge.py gen --style pop --title 歌名 --lyrics-file input\lyrics\歌名.txt .\.venv\Scripts\python.exe songforge.py gen --style pop --title 灯还亮着 --lyrics-file input\lyrics\pop_灯还亮着.txt
.\.venv\Scripts\python.exe songforge.py qc --run gen-<时间戳> --baseline .\.venv\Scripts\python.exe songforge.py qc --run gen-<时间戳> --baseline
.\.venv\Scripts\python.exe songforge.py prune --days 7 # 清过期 tmp
.\.venv\Scripts\python.exe songforge.py prune --days 7 --dry-run
``` ```
`qc` 必须带 `--baseline`。完整 Agent 规则与「何时发问」见 [`AGENTS.md`](AGENTS.md)。 `qc` 必须带 `--baseline`。样例歌词见 `input/lyrics/`。Agent 规则见 [`AGENTS.md`](AGENTS.md)。
## 硬约束(摘要) ## 硬约束(摘要)
| 项 | 值 | | 项 | 值 |
|----|-----| |----|-----|
| 模型 | `ace_step_1.5_turbo_swl_fused` + `swl_voice_best_ep257`(0.5/0.5) | | 模型 | 见上表(0.5/0.5 LoRA) |
| 采样 | shift=5.5 / steps=28 / cfg=2.0 / temperature=0.68 | | 采样 | shift=5.5 / steps=28 / cfg=2.0 / temperature=0.68 |
| 时长 | 完整歌 150–200 秒 | | 时长 | 完整歌 150–200 秒 |
| 前奏 | 8 小节器乐 hook;CLI 自动注入 | | 前奏 | 8 小节器乐 hook;CLI 自动注入 |
## 远程 ## 远程(双 remote)
- Gitea: https://git.nnrc.ren/swl/songforge | 名 | URL | 用途 |
- GitHub: https://github.com/lsuen/songforge |----|-----|------|
| `gitea` | https://git.nnrc.ren/swl/songforge | 常用推送 |
| `origin` / GitHub | https://github.com/lsuen/songforge | 公开镜像 |
```powershell
git remote -v
git push gitea main
git push origin main # 或经 ghfast
```
Agent 推送前先问用户推哪个 remote;**不要**推 `archive/*`、`.secrets`、`local.toml`。
## License ## License
+60 -1
View File
@@ -1,9 +1,10 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
"""CLI 命令实现:doctor / styles / target / check / gen / runs / show。""" """CLI 命令实现:doctor / styles / target / check / gen / runs / show / prune。"""
import datetime import datetime
import json import json
import os import os
import random import random
import shutil
from ..config import loader from ..config import loader
from ..core import catalog as catmod from ..core import catalog as catmod
@@ -83,12 +84,23 @@ def cmd_doctor(args):
lora = pmod.model_path(root, "loras", cfg["model"]["lora"]) lora = pmod.model_path(root, "loras", cfg["model"]["lora"])
tpl = layout["template"] tpl = layout["template"]
rc = 0 rc = 0
missing = []
for label, p in (("checkpoint", ck), ("lora", lora), for label, p in (("checkpoint", ck), ("lora", lora),
("template", tpl)): ("template", tpl)):
good = os.path.exists(p) good = os.path.exists(p)
print(" [%s] %-11s %s" % ("ok" if good else "FAIL", label, p)) print(" [%s] %-11s %s" % ("ok" if good else "FAIL", label, p))
if not good: if not good:
rc = 2 rc = 2
missing.append((label, p))
if missing:
print(" ----")
print(" 缺模型/模板时:")
print(" 1) 确认 config/local.toml 的 paths.comfy_root 指向 ComfyUI 安装根")
print(" 2) 放入锁定文件(文件名需一致,或改 default.toml 并接受声线变化):")
print(" models/checkpoints/%s" % cfg["model"]["checkpoint"])
print(" models/loras/%s" % cfg["model"]["lora"])
print(" 3) 本仓默认声线 LoRA 为作者定制;他人需自备同名或替换")
print(" 4) 路径不清楚 → 问用户,不要猜盘符")
if cfg["model"]["checkpoint"] in cfg["model"]["denied_checkpoints"]: if cfg["model"]["checkpoint"] in cfg["model"]["denied_checkpoints"]:
print(" [FAIL] 当前 checkpoint 在禁用名单里:%s" print(" [FAIL] 当前 checkpoint 在禁用名单里:%s"
% cfg["model"]["checkpoint"]) % cfg["model"]["checkpoint"])
@@ -438,3 +450,50 @@ def cmd_catalog(args):
d.get("bpm"), d.get("duration"), d.get("bpm"), d.get("duration"),
d.get("qc_verdict") or "-", d.get("style"))) d.get("qc_verdict") or "-", d.get("style")))
return 0 return 0
def cmd_prune(args):
"""清理过期 data/tmp(默认不动 runs / 音频)。"""
cfg = loader.load({})
tmp_root = pmod.under_pkg(cfg["paths"]["tmp_dir"])
if not os.path.isdir(tmp_root):
print("tmp 目录不存在: %s" % tmp_root)
return 0
keep_days = int(args.days)
cutoff = datetime.date.today() - datetime.timedelta(days=keep_days)
removed = []
skipped = []
for name in sorted(os.listdir(tmp_root)):
path = os.path.join(tmp_root, name)
if not os.path.isdir(path):
continue
# YYYY-MM-DD 或 _migrated
day = None
try:
day = datetime.date.fromisoformat(name)
except ValueError:
if name.startswith("_") and args.include_migrated:
day = cutoff # 当作过期
else:
skipped.append(name)
continue
if day >= cutoff:
continue
if args.dry_run:
removed.append(path + " (dry-run)")
continue
shutil.rmtree(path, ignore_errors=False)
removed.append(path)
print("tmp_root %s" % tmp_root)
print("keep_days %d (删除早于 %s 的日目录)" % (keep_days, cutoff))
if not removed:
print("无需清理")
else:
for p in removed:
print(" - %s" % p)
print("已处理 %d 项%s" % (len(removed),
"(dry-run 未删)" if args.dry_run else ""))
if skipped:
print("跳过非日期目录: %s" % ", ".join(skipped[:8]))
print("未动 data/runs 与 Comfy 音频输出")
return 0
+13 -2
View File
@@ -2,7 +2,8 @@
"""songforge CLI 驱动(薄调度层)。 """songforge CLI 驱动(薄调度层)。
命令: 命令:
doctor / styles / target / check / gen / runs / show / catalog / qc / selftest doctor / styles / target / check / gen / runs / show / catalog /
prune / qc / selftest
设计约束:所有业务参数来自配置/命令行,本文件不写死任何模型或采样常量。 设计约束:所有业务参数来自配置/命令行,本文件不写死任何模型或采样常量。
实现见 common / assemble / commands / selftest / qc。 实现见 common / assemble / commands / selftest / qc。
@@ -19,7 +20,7 @@ def build_parser():
p = argparse.ArgumentParser( p = argparse.ArgumentParser(
prog="songforge", prog="songforge",
description="ACE-Step 生歌引擎。配置优先级:命令行 > 环境变量 > " description="ACE-Step 生歌引擎。配置优先级:命令行 > 环境变量 > "
"local.json > default.json") "local.toml > default.toml")
p.add_argument("--server", help="ComfyUI 地址") p.add_argument("--server", help="ComfyUI 地址")
p.add_argument("--log-level", default="info", p.add_argument("--log-level", default="info",
choices=["debug", "info", "warn", "error"]) choices=["debug", "info", "warn", "error"])
@@ -135,6 +136,16 @@ def build_parser():
help="从 data/runs 扫描重建 catalog.jsonl") help="从 data/runs 扫描重建 catalog.jsonl")
cat.set_defaults(func=cmds.cmd_catalog) cat.set_defaults(func=cmds.cmd_catalog)
pr = sub.add_parser("prune", help="清理过期 data/tmp(不动 runs/音频)")
pr.add_argument("--days", type=int, default=7,
help="保留最近 N 天的 tmp 日目录(默认 7)")
pr.add_argument("--dry-run", dest="dry_run", action="store_true",
help="只列出将删除的目录")
pr.add_argument("--include-migrated", dest="include_migrated",
action="store_true",
help="一并删除 tmp/_migrated 等非日期目录")
pr.set_defaults(func=cmds.cmd_prune)
from .qc import add_qc from .qc import add_qc
add_qc(sub) add_qc(sub)
return p return p
+2
View File
@@ -47,6 +47,8 @@ doctor → target → 写歌词 → check → gen → qc --baseline
| 生成 | `gen …` | 看 `data/runs/<日>/<id>/run.log`;服务挂了再 `doctor` | | 生成 | `gen …` | 看 `data/runs/<日>/<id>/run.log`;服务挂了再 `doctor` |
| 验收 | `qc --run gen-… --baseline` | FAIL 修歌/重跑,不放宽阈值 | | 验收 | `qc --run gen-… --baseline` | FAIL 修歌/重跑,不放宽阈值 |
| 索引 | `catalog` / `catalog --rebuild` | 查歌名/风格/BPM/QC 结论 | | 索引 | `catalog` / `catalog --rebuild` | 查歌名/风格/BPM/QC 结论 |
| 清理 | `prune --days 7` | 删过期 `data/tmp`,不动 runs/音频 |
| 一键体检 | `.\scripts\doctor_all.ps1` | venv + local.toml + doctor |
`qc` 报告里的 `style_fam` / `style_conf` 是弱参考,**不会**单独导致 FAIL。对外不要把 PASS 说成「风格已验证」——当前风格物理保证主要是实测 BPM 对齐。 `qc` 报告里的 `style_fam` / `style_conf` 是弱参考,**不会**单独导致 FAIL。对外不要把 PASS 说成「风格已验证」——当前风格物理保证主要是实测 BPM 对齐。
+8 -4
View File
@@ -1,11 +1,15 @@
# 歌词投放目录 # 歌词投放目录
新歌词 `.txt` 放这里,生成时: 新歌词 `.txt` 放这里。仓内样例(可直接 `check` / 冒烟):
| 文件 | 风格建议 |
|------|----------|
| `pop_灯还亮着.txt` | `--style pop` |
| `hiphop_楼下那条街.txt` | `--style urbanhiphop` 或 hiphop 系 |
| `punk_去它妈的班.txt` | `--style punk` |
```powershell ```powershell
cd D:\CodeFiles\Thinking\ComfyUI-MiniMaxH3\user\default\workflows .\.venv\Scripts\python.exe songforge.py check --style pop --title 灯还亮着 --lyrics-file input\lyrics\pop_灯还亮着.txt --bpm 104 --dur 180
python songforge\songforge.py check --style pop --title 歌名 --lyrics-file songforge\input\lyrics\歌名.txt
``` ```
历史素材库在 `../_archive/lyrics_corpus/`(只读)。
不要手写 `[intro]` / tags,CLI 会注入。 不要手写 `[intro]` / tags,CLI 会注入。
+45
View File
@@ -0,0 +1,45 @@
# One-shot setup check: venv -> local.toml -> doctor -> next steps
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent (Split-Path -Parent $MyInvocation.MyCommand.Path)
Set-Location $Root
Write-Host "== songforge doctor_all =="
Write-Host "pkg: $Root"
$py = Join-Path $Root ".venv\Scripts\python.exe"
if (-not (Test-Path $py)) {
Write-Host "No .venv — running setup_venv.ps1 ..."
& powershell -ExecutionPolicy Bypass -File (Join-Path $Root "scripts\setup_venv.ps1")
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
}
$local = Join-Path $Root "config\local.toml"
if (-not (Test-Path $local)) {
$ex = Join-Path $Root "config\local.toml.example"
Copy-Item $ex $local
Write-Host ""
Write-Host "CREATED config\local.toml from example."
Write-Host "Edit comfy_root / server / qc_python, start ComfyUI, then re-run:"
Write-Host " .\scripts\doctor_all.ps1"
exit 2
}
Write-Host "Using: $py"
& $py (Join-Path $Root "songforge.py") doctor @args
$code = $LASTEXITCODE
Write-Host ""
if ($code -eq 0) {
Write-Host "NEXT:"
Write-Host " .\.venv\Scripts\python.exe songforge.py target --style pop --bpm 104 --dur 180"
Write-Host " .\.venv\Scripts\python.exe songforge.py check --style pop --title demo --lyrics-file input\lyrics\pop_灯还亮着.txt"
Write-Host " .\.venv\Scripts\python.exe songforge.py gen --style pop --title demo --lyrics-file input\lyrics\pop_灯还亮着.txt"
} else {
Write-Host "doctor FAILED (exit $code)."
Write-Host "Common fixes:"
Write-Host " - Start ComfyUI (default http://127.0.0.1:8188)"
Write-Host " - Fix config\local.toml paths.comfy_root"
Write-Host " - Put locked checkpoint + LoRA under <comfy_root>\models\ (see README)"
Write-Host " - Ask the user if paths are unknown (Agents: see AGENTS.md)"
}
exit $code