verify.sh 如何把“完成”变成证据 深度解析
这个子页面拆 verify runner 与测试套件。Agentsmith 的强约束是:每类工作都需要一个真实 verify 入口,人和 agent 都跑同一条命令,而不是在会话总结里宣称“应该没问题”。
导入链与契约
`verify.sh` 的契约和软件开发 profile 是互相咬合的。core/20-principle-rules.md:13、core/20-principle-rules.md:23、core/20-principle-rules.md:35 先规定证据、端到端链路和完成前检查,再由 profiles/software-dev.md:11、profiles/software-dev.md:15、profiles/software-dev.md:19、profiles/software-dev.md:46 定义质量地板,最后由 scripts/verify.sh:17、scripts/verify.sh:58、scripts/verify.sh:82 定义执行格式。
17# .harness/verify.conf format — one phase per line, "Label :: shell command": 18# build :: <your build command> 19# tests :: <your test command> 20# lint :: <your lint command> 21# links :: <your link/spell/render check> 22# Lines starting with # and blank lines are ignored. See verify.conf.example. 23 24set -euo pipefail 25 26SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" 27ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" 28CONF="$ROOT_DIR/.harness/verify.conf" 29
核心实现,逐段拆
▸Phase parser 与 first-failure 语义
解析逻辑在 scripts/verify.sh:60、scripts/verify.sh:63、scripts/verify.sh:67、scripts/verify.sh:71,执行逻辑在 scripts/verify.sh:84、scripts/verify.sh:87、scripts/verify.sh:88、scripts/verify.sh:94。这让每个项目自己定义 build/test/lint,但 runner 保证顺序、过滤和失败语义一致;它对应的规则依据是 core/20-principle-rules.md:35 到 core/20-principle-rules.md:37 的“先跑完整检查,再声明成功”。
58# Parse "Label :: command" lines (skip comments/blanks). 59LABELS=(); CMDS=() 60while IFS= read -r line; do 61 [ -z "${line//[[:space:]]/}" ] && continue 62 case "$line" in \#*) continue ;; esac 63 label="${line%%::*}"; cmd="${line#*::}" 64 label="$(echo "$label" | sed 's/[[:space:]]*$//;s/^[[:space:]]*//')" 65 cmd="$(echo "$cmd" | sed 's/^[[:space:]]*//')" 66 [ -z "$cmd" ] && continue 67 if [ -n "$ONLY" ] && [[ "$label" != *"$ONLY"* ]]; then continue; fi 68 LABELS+=("$label"); CMDS+=("$cmd") 69done < "$CONF" 70 71TOTAL=${#LABELS[@]} 72if [ "$TOTAL" -eq 0 ]; then 73 echo "${YELLOW}No matching phases.${RESET}"; exit 0 74fi 75 76if $LIST_ONLY; then 77 echo "${BOLD}Configured phases:${RESET}" 78 for i in "${!LABELS[@]}"; do printf ' %d. %s :: %s\n' "$((i+1))" "${LABELS[$i]}" "${CMDS[$i]}"; done 79 exit 0 80fi 81 82cd "$ROOT_DIR" 83START=$SECONDS 84for i in "${!LABELS[@]}"; do 85 echo; echo "${BOLD}${CYAN}[$((i+1))/$TOTAL] ${LABELS[$i]}${RESET}" 86 echo " ${CMDS[$i]}" 87 if ! bash -c "${CMDS[$i]}"; then 88 echo; echo "${BOLD}${RED}verify.sh: phase '${LABELS[$i]}' failed.${RESET}" 89 echo "${YELLOW}Run that command directly to iterate, then re-run verify.sh.${RESET}" 90 exit 1 91 fi 92done 93 94echo; echo "${BOLD}${GREEN}verify.sh: all $TOTAL phase(s) passed in $((SECONDS-START))s.${RESET}"
▸Profile 为什么要求 full verify
软件开发 profile 在 profiles/software-dev.md:21、profiles/software-dev.md:23、profiles/software-dev.md:35、profiles/software-dev.md:39、profiles/software-dev.md:40 区分自动化绿色和真实路径运行,避免只跑局部测试。它也是对 core/20-principle-rules.md:23、core/20-principle-rules.md:26、core/20-principle-rules.md:28 的具体化:只验证一层不够,必须追到用户真正看到的最后一层。
19"Verified" has two distinct gates — pass both: 20 21- **Within a layer (automated):** build, type-check, lint, unit/integration 22 tests pass. These prove the code is internally correct. 23- **Across layers (exercised):** the changed path was actually run — a real 24 invocation, a request through the running service, a click-through in the UI. 25 Automated green does NOT prove the feature works end to end. Trace one 26 concrete value through every layer and every fan-out consumer it reaches 27 (each route, caller, callback, config variant) before claiming done (R3). 28 29"It compiles and the unit test passes" is within-a-layer evidence only. If a 30human could click a button and see it break, you haven't verified it. 31 32### Quality gates 33Before calling code done, tick each — "deferred: reason" is allowed, silence is not: 34 35- [ ] `<your build cmd>` succeeds (no warnings you introduced) 36- [ ] `<your typecheck cmd>` clean 37- [ ] `<your lint cmd>` clean 38- [ ] new/changed behavior has a test that failed before the fix (R2) 39- [ ] the **full** test suite passes, not just the new test (R5) 40- [ ] the changed path was **run for real** once (CLI invocation / live request / 41 browser click-through), including every fan-out consumer (R3) 42- [ ] docs, changelog, help text, and inline comments that the change made wrong 43 are fixed in this same unit (R6) 44- [ ] a defect ticket exists for anything found-but-not-fixed (R7) 45 46Single entry point: run `scripts/verify.sh` (it should chain build → typecheck → 47lint → tests). The actual commands live in `.harness/verify.conf` so the script 48and the human stay in sync — edit the conf, not the call sites. If `verify.sh` 49doesn't exist yet, create it to wrap the conf; never scatter raw commands across
关键流程
▸Phase 表如何变成证据
从工程角度看,这个设计的好处是把“怎么证明”外置到配置,把“如何按顺序执行并报告”固定在脚本。core/20-principle-rules.md:13、core/20-principle-rules.md:17、core/20-principle-rules.md:35 给出证据标准,scripts/verify.sh:52、scripts/verify.sh:76、scripts/verify.sh:87、profiles/software-dev.md:46 使 agent 和人类使用同一入口。
边界 / 失败模式
scripts/verify.sh:87使用 bash -c,配置文件必须被当作代码审查。scripts/verify.sh:72在没有匹配 phase 时返回 0,这对 `--only` 调试友好,但 CI 不应只跑窄 tag。profiles/software-dev.md:40要求真实路径运行,这部分无法完全由 verify.sh 自动推断。