Add doctor_all, prune, model docs, and selftest CI.
selftest / selftest (push) Canceled after 0s

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
swl
2026-10-10 02:06:54 +08:00
co-authored by Cursor
parent 1de2654858
commit dbfb9b52f9
8 changed files with 217 additions and 30 deletions
+37
View File
@@ -0,0 +1,37 @@
name: selftest
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
selftest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install
run: |
python -m pip install -U pip
pip install -r requirements.txt
- name: Stub ComfyUI layout
run: |
mkdir -p models/checkpoints models/loras output/MyMusic user/default/workflows/bak
touch models/checkpoints/ace_step_1.5_turbo_swl_fused.safetensors
touch models/loras/swl_voice_best_ep257.safetensors
touch user/default/workflows/song_rules.py
touch "user/default/workflows/urbanhiphop-身体先听懂.json"
echo stub > user/default/workflows/bak/ci-stub
- name: Layout for package import
run: |
mkdir -p /tmp/sfwrap
ln -s "$GITHUB_WORKSPACE" /tmp/sfwrap/songforge
- name: selftest
working-directory: /tmp/sfwrap
run: python songforge/songforge.py selftest
env:
SONGFORGE_COMFY_ROOT: ${{ github.workspace }}
+9 -7
View File
@@ -55,12 +55,9 @@ git clone https://git.nnrc.ren/swl/songforge.git
# 或 https://github.com/lsuen/songforge.git
cd songforge
.\scripts\setup_venv.ps1
copy config\local.toml.example config\local.toml
# 编辑 local.toml:engine.server / paths.comfy_root / qc_python / music_dir
# 用户需先启动本机 ComfyUI
.\.venv\Scripts\python.exe songforge.py doctor
.\scripts\doctor_all.ps1
# 缺 local.toml 时会复制 example 并退出;编辑并启动 ComfyUI 后再跑
# 模型文件名见 README「需要的模型」;没有作者 LoRA 时先问用户
```
入口(在**仓根**):
@@ -156,17 +153,22 @@ doctor → target → 写歌词 → check → gen → qc --baseline
|------|------|
| 无 local.toml | **问用户**路径后创建并填写 |
| doctor FAIL 服务 | 请用户启动 ComfyUI;核对 server |
| doctor FAIL 模型 | **问**模型是否在该 comfy_root |
| doctor FAIL 模型 | **问**模型是否在该 comfy_root;对照 README 文件名 |
| check 密度失败 | 改歌词到 target 区间 |
| gen 提交失败 | `selftest`;必要时 `--dry-run` |
| qc voice_nn=null | 补 `--baseline` |
| qc FAIL | 修歌/重 gen,**不**改阈值;反复失败则**问用户**听哪一版 |
| tmp 太肥 | `prune --days 7`(不动 runs/音频) |
| CLI 像坏了 | `selftest`;仍异常则**问用户**并贴 log |
| 要 push | **问**推 `gitea` 还是 GitHub;勿推 secrets/archive |
```powershell
.\.venv\Scripts\python.exe songforge.py selftest
.\.venv\Scripts\python.exe songforge.py prune --days 7 --dry-run
```
冒烟歌词:`input/lyrics/pop_灯还亮着.txt`。
---
## 8. 验收清单
+43 -16
View File
@@ -12,6 +12,14 @@
> 给 Agent:读完 `AGENTS.md` 再动手。缺 `local.toml`、Comfy 没起、风格/歌名不清、想改模型或阈值时——**先问用户,不要猜**。
## 一键体检
```powershell
.\scripts\doctor_all.ps1
```
会:缺 `.venv` 则创建 → 缺 `local.toml` 则从 example 复制并提示编辑 → 跑 `doctor` → 打印下一步。
## 换目录也能用(推荐)
```powershell
@@ -20,23 +28,30 @@ git clone https://git.nnrc.ren/swl/songforge.git
# 或:git clone https://github.com/lsuen/songforge.git
cd songforge
.\scripts\setup_venv.ps1
copy config\local.toml.example config\local.toml
# 编辑 local.toml:
# [engine] server = "http://127.0.0.1:8188"
# [paths] comfy_root / qc_python / music_dir
# 先启动本机 ComfyUI,再:
.\.venv\Scripts\python.exe songforge.py doctor
.\scripts\doctor_all.ps1
# 按提示编辑 config\local.toml 后,启动 ComfyUI,再跑一次 doctor_all
```
不必把本仓放进 ComfyUI 的 `user/default/workflows`。
## 需要的模型(ComfyUI 侧)
在 `local.toml` 的 `comfy_root` 下准备:
| 类型 | 文件名 | 目录 |
|------|--------|------|
| Checkpoint | `ace_step_1.5_turbo_swl_fused.safetensors` | `models/checkpoints/` |
| LoRA | `swl_voice_best_ep257.safetensors` | `models/loras/` |
- 默认 LoRA 是**作者定制声线**;别人没有同文件时 `doctor` 会 FAIL。
- 可换成自己的 LoRA,并改 `config/default.toml` 的 `[model]`(声线会变;不要用禁用名单里的 YuE2/aio)。
- 缺文件时 `doctor` 会打印人话排查步骤。
## 两个 Python 环境
| 环境 | 用途 |
|------|------|
| **本仓 `.venv`** | CLI:`doctor` / `check` / `gen` / `catalog` |
| **本仓 `.venv`** | CLI:`doctor` / `check` / `gen` / `catalog` / `prune` |
| **`qc_python`** | QC 音频栈(可继续用 ComfyUI 的 venv) |
## 配置:TOML vs JSON
@@ -55,26 +70,38 @@ copy config\local.toml.example config\local.toml
```powershell
.\.venv\Scripts\python.exe songforge.py target --style pop --bpm 104 --dur 180
.\.venv\Scripts\python.exe songforge.py check --style pop --title 歌名 --lyrics-file input\lyrics\歌名.txt
.\.venv\Scripts\python.exe songforge.py gen --style pop --title 歌名 --lyrics-file input\lyrics\歌名.txt
.\.venv\Scripts\python.exe songforge.py check --style pop --title 灯还亮着 --lyrics-file input\lyrics\pop_灯还亮着.txt
.\.venv\Scripts\python.exe songforge.py gen --style pop --title 灯还亮着 --lyrics-file input\lyrics\pop_灯还亮着.txt
.\.venv\Scripts\python.exe songforge.py qc --run gen-<时间戳> --baseline
.\.venv\Scripts\python.exe songforge.py prune --days 7 # 清过期 tmp
.\.venv\Scripts\python.exe songforge.py prune --days 7 --dry-run
```
`qc` 必须带 `--baseline`。完整 Agent 规则与「何时发问」见 [`AGENTS.md`](AGENTS.md)。
`qc` 必须带 `--baseline`。样例歌词见 `input/lyrics/`。Agent 规则见 [`AGENTS.md`](AGENTS.md)。
## 硬约束(摘要)
| 项 | 值 |
|----|-----|
| 模型 | `ace_step_1.5_turbo_swl_fused` + `swl_voice_best_ep257`(0.5/0.5) |
| 模型 | 见上表(0.5/0.5 LoRA) |
| 采样 | shift=5.5 / steps=28 / cfg=2.0 / temperature=0.68 |
| 时长 | 完整歌 150–200 秒 |
| 前奏 | 8 小节器乐 hook;CLI 自动注入 |
## 远程
## 远程(双 remote)
- Gitea: https://git.nnrc.ren/swl/songforge
- GitHub: https://github.com/lsuen/songforge
| 名 | URL | 用途 |
|----|-----|------|
| `gitea` | https://git.nnrc.ren/swl/songforge | 常用推送 |
| `origin` / GitHub | https://github.com/lsuen/songforge | 公开镜像 |
```powershell
git remote -v
git push gitea main
git push origin main # 或经 ghfast
```
Agent 推送前先问用户推哪个 remote;**不要**推 `archive/*`、`.secrets`、`local.toml`。
## License
+60 -1
View File
@@ -1,9 +1,10 @@
# -*- coding: utf-8 -*-
"""CLI 命令实现:doctor / styles / target / check / gen / runs / show。"""
"""CLI 命令实现:doctor / styles / target / check / gen / runs / show / prune。"""
import datetime
import json
import os
import random
import shutil
from ..config import loader
from ..core import catalog as catmod
@@ -83,12 +84,23 @@ def cmd_doctor(args):
lora = pmod.model_path(root, "loras", cfg["model"]["lora"])
tpl = layout["template"]
rc = 0
missing = []
for label, p in (("checkpoint", ck), ("lora", lora),
("template", tpl)):
good = os.path.exists(p)
print(" [%s] %-11s %s" % ("ok" if good else "FAIL", label, p))
if not good:
rc = 2
missing.append((label, p))
if missing:
print(" ----")
print(" 缺模型/模板时:")
print(" 1) 确认 config/local.toml 的 paths.comfy_root 指向 ComfyUI 安装根")
print(" 2) 放入锁定文件(文件名需一致,或改 default.toml 并接受声线变化):")
print(" models/checkpoints/%s" % cfg["model"]["checkpoint"])
print(" models/loras/%s" % cfg["model"]["lora"])
print(" 3) 本仓默认声线 LoRA 为作者定制;他人需自备同名或替换")
print(" 4) 路径不清楚 → 问用户,不要猜盘符")
if cfg["model"]["checkpoint"] in cfg["model"]["denied_checkpoints"]:
print(" [FAIL] 当前 checkpoint 在禁用名单里:%s"
% cfg["model"]["checkpoint"])
@@ -438,3 +450,50 @@ def cmd_catalog(args):
d.get("bpm"), d.get("duration"),
d.get("qc_verdict") or "-", d.get("style")))
return 0
def cmd_prune(args):
"""清理过期 data/tmp(默认不动 runs / 音频)。"""
cfg = loader.load({})
tmp_root = pmod.under_pkg(cfg["paths"]["tmp_dir"])
if not os.path.isdir(tmp_root):
print("tmp 目录不存在: %s" % tmp_root)
return 0
keep_days = int(args.days)
cutoff = datetime.date.today() - datetime.timedelta(days=keep_days)
removed = []
skipped = []
for name in sorted(os.listdir(tmp_root)):
path = os.path.join(tmp_root, name)
if not os.path.isdir(path):
continue
# YYYY-MM-DD 或 _migrated
day = None
try:
day = datetime.date.fromisoformat(name)
except ValueError:
if name.startswith("_") and args.include_migrated:
day = cutoff # 当作过期
else:
skipped.append(name)
continue
if day >= cutoff:
continue
if args.dry_run:
removed.append(path + " (dry-run)")
continue
shutil.rmtree(path, ignore_errors=False)
removed.append(path)
print("tmp_root %s" % tmp_root)
print("keep_days %d (删除早于 %s 的日目录)" % (keep_days, cutoff))
if not removed:
print("无需清理")
else:
for p in removed:
print(" - %s" % p)
print("已处理 %d 项%s" % (len(removed),
"(dry-run 未删)" if args.dry_run else ""))
if skipped:
print("跳过非日期目录: %s" % ", ".join(skipped[:8]))
print("未动 data/runs 与 Comfy 音频输出")
return 0
+13 -2
View File
@@ -2,7 +2,8 @@
"""songforge CLI 驱动(薄调度层)。
命令:
doctor / styles / target / check / gen / runs / show / catalog / qc / selftest
doctor / styles / target / check / gen / runs / show / catalog /
prune / qc / selftest
设计约束:所有业务参数来自配置/命令行,本文件不写死任何模型或采样常量。
实现见 common / assemble / commands / selftest / qc。
@@ -19,7 +20,7 @@ def build_parser():
p = argparse.ArgumentParser(
prog="songforge",
description="ACE-Step 生歌引擎。配置优先级:命令行 > 环境变量 > "
"local.json > default.json")
"local.toml > default.toml")
p.add_argument("--server", help="ComfyUI 地址")
p.add_argument("--log-level", default="info",
choices=["debug", "info", "warn", "error"])
@@ -135,6 +136,16 @@ def build_parser():
help="从 data/runs 扫描重建 catalog.jsonl")
cat.set_defaults(func=cmds.cmd_catalog)
pr = sub.add_parser("prune", help="清理过期 data/tmp(不动 runs/音频)")
pr.add_argument("--days", type=int, default=7,
help="保留最近 N 天的 tmp 日目录(默认 7)")
pr.add_argument("--dry-run", dest="dry_run", action="store_true",
help="只列出将删除的目录")
pr.add_argument("--include-migrated", dest="include_migrated",
action="store_true",
help="一并删除 tmp/_migrated 等非日期目录")
pr.set_defaults(func=cmds.cmd_prune)
from .qc import add_qc
add_qc(sub)
return p
+2
View File
@@ -47,6 +47,8 @@ doctor → target → 写歌词 → check → gen → qc --baseline
| 生成 | `gen …` | 看 `data/runs/<日>/<id>/run.log`;服务挂了再 `doctor` |
| 验收 | `qc --run gen-… --baseline` | FAIL 修歌/重跑,不放宽阈值 |
| 索引 | `catalog` / `catalog --rebuild` | 查歌名/风格/BPM/QC 结论 |
| 清理 | `prune --days 7` | 删过期 `data/tmp`,不动 runs/音频 |
| 一键体检 | `.\scripts\doctor_all.ps1` | venv + local.toml + doctor |
`qc` 报告里的 `style_fam` / `style_conf` 是弱参考,**不会**单独导致 FAIL。对外不要把 PASS 说成「风格已验证」——当前风格物理保证主要是实测 BPM 对齐。
+8 -4
View File
@@ -1,11 +1,15 @@
# 歌词投放目录
新歌词 `.txt` 放这里,生成时:
新歌词 `.txt` 放这里。仓内样例(可直接 `check` / 冒烟):
| 文件 | 风格建议 |
|------|----------|
| `pop_灯还亮着.txt` | `--style pop` |
| `hiphop_楼下那条街.txt` | `--style urbanhiphop` 或 hiphop 系 |
| `punk_去它妈的班.txt` | `--style punk` |
```powershell
cd D:\CodeFiles\Thinking\ComfyUI-MiniMaxH3\user\default\workflows
python songforge\songforge.py check --style pop --title 歌名 --lyrics-file songforge\input\lyrics\歌名.txt
.\.venv\Scripts\python.exe songforge.py check --style pop --title 灯还亮着 --lyrics-file input\lyrics\pop_灯还亮着.txt --bpm 104 --dur 180
```
历史素材库在 `../_archive/lyrics_corpus/`(只读)。
不要手写 `[intro]` / tags,CLI 会注入。
+45
View File
@@ -0,0 +1,45 @@
# One-shot setup check: venv -> local.toml -> doctor -> next steps
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent (Split-Path -Parent $MyInvocation.MyCommand.Path)
Set-Location $Root
Write-Host "== songforge doctor_all =="
Write-Host "pkg: $Root"
$py = Join-Path $Root ".venv\Scripts\python.exe"
if (-not (Test-Path $py)) {
Write-Host "No .venv — running setup_venv.ps1 ..."
& powershell -ExecutionPolicy Bypass -File (Join-Path $Root "scripts\setup_venv.ps1")
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
}
$local = Join-Path $Root "config\local.toml"
if (-not (Test-Path $local)) {
$ex = Join-Path $Root "config\local.toml.example"
Copy-Item $ex $local
Write-Host ""
Write-Host "CREATED config\local.toml from example."
Write-Host "Edit comfy_root / server / qc_python, start ComfyUI, then re-run:"
Write-Host " .\scripts\doctor_all.ps1"
exit 2
}
Write-Host "Using: $py"
& $py (Join-Path $Root "songforge.py") doctor @args
$code = $LASTEXITCODE
Write-Host ""
if ($code -eq 0) {
Write-Host "NEXT:"
Write-Host " .\.venv\Scripts\python.exe songforge.py target --style pop --bpm 104 --dur 180"
Write-Host " .\.venv\Scripts\python.exe songforge.py check --style pop --title demo --lyrics-file input\lyrics\pop_灯还亮着.txt"
Write-Host " .\.venv\Scripts\python.exe songforge.py gen --style pop --title demo --lyrics-file input\lyrics\pop_灯还亮着.txt"
} else {
Write-Host "doctor FAILED (exit $code)."
Write-Host "Common fixes:"
Write-Host " - Start ComfyUI (default http://127.0.0.1:8188)"
Write-Host " - Fix config\local.toml paths.comfy_root"
Write-Host " - Put locked checkpoint + LoRA under <comfy_root>\models\ (see README)"
Write-Host " - Ask the user if paths are unknown (Agents: see AGENTS.md)"
}
exit $code