Просмотр исходного кода

test(skills): section-map drift gates for summon.py and svg-brand-tint index.html

Add a mechanical section-map drift gate to each skill's tests/run.sh so the
top-of-file section map carried by two deliberately-large single files
cannot silently rot as they grow. Nothing enforced this before.

- svg-brand-tint-ops/assets/index.html: bidirectional name match between the
  guard comment's `Sections (grep "=== NAME ===")` list and the body
  `// === NAME ===` markers — forward (every listed section has a marker)
  and reverse (every marker is listed) — plus an empty-parse guard.
- summon/scripts/summon.py: the docstring `Sections:` list and the body
  `# ===` banner pairs use different labels for several sections
  (DESIGN(term)/DESIGN, Modes/Toolbox modes, CLI entry/Main) and the map
  lists one banner-less concept (Transcript/Distill), so strict name-equality
  would false-fail on the current file. The gate instead pins both the
  docstring section count (14) and the body banner count (13) — the
  mechanical structural-sync check that stays green yet fails every
  add/remove mutation. The body-banner parser keys off the `# ===…===` header
  pairs; the docstring parser keys off the `Sections:` line.

Empty-parse-is-failure rule: if a map/marker format change on either side
yields zero parsed names, the gate hard-FAILs (never a silent pass) — that
is exactly the rot mode this guard exists to catch.

Verified: both suites green (svg 25/0; summon 38 behavioural + 2 gate / 0);
mutation-tested both directions on temp copies — deleting a marker fails the
forward/body assertion, adding a bogus marker fails the reverse/body
assertion, and zeroing the parse fails the empty-parse guard — all with
exit 1; sources restored and only the two run.sh files are modified.

Co-Authored-By: Claude <noreply@anthropic.com>
0xDarkMatter 1 неделя назад
Родитель
Сommit
d2fa4fab1d
2 измененных файлов с 127 добавлено и 3 удалено
  1. 82 3
      skills/summon/tests/run.sh
  2. 45 0
      skills/svg-brand-tint-ops/tests/run.sh

+ 82 - 3
skills/summon/tests/run.sh

@@ -11,8 +11,11 @@
 # no real LLM call is ever made by this suite), the pick --json inventory
 # envelope, and the in-chat picker asset (present + cited from SKILL.md).
 #
-# All checks live in test_summon.py so its pass/fail summary IS the whole
-# suite — no shell-level checks that could fail outside the counter.
+# The behavioural checks live in test_summon.py — its pass/fail summary is the
+# primary signal. One shell-level check also runs after it (below): a
+# section-map drift gate that pins the docstring 'Sections:' list against the
+# body's `# ===` banner headers, so the deliberately-single-file script's map
+# cannot silently rot as it grows.
 #
 # Usage:   bash tests/run.sh
 # Exit:    0 all pass, 1 one or more failures
@@ -28,4 +31,80 @@ for c in python python3 py; do
 done
 [[ -z "$PYTHON" ]] && { echo "no working python found" >&2; exit 1; }
 
-exec "$PYTHON" "$HERE/test_summon.py"
+# Run the full behavioural suite, then fall through to the shell-level
+# section-map drift gate (we deliberately do NOT `exec` the python here — the
+# gate must run afterwards and contribute to the combined exit code).
+SUMMON_PY_RC=0
+"$PYTHON" "$HERE/test_summon.py" || SUMMON_PY_RC=$?
+
+# --- section-map drift gate (summon.py docstring 'Sections:' ↔ # === banners) ---
+# summon.py is deliberately a single multi-thousand-line file
+# (docs/SKILL-RESOURCE-PROTOCOL.md: skill scripts ship as self-contained
+# portable units — do not split). Its module docstring carries a `Sections:`
+# map of the `# ===` banner headers so the file stays navigable. This gate
+# pins BOTH the docstring section count and the body banner count, so a
+# section added or removed on either side fails the build until the map is
+# reconciled. An empty parse on either side is a hard FAIL (never a silent
+# pass) — that is the rot mode this guard exists to catch: a docstring/map
+# format change that yields zero names.
+#
+# Strict name-equality matching is intentionally NOT used: the docstring and
+# the banner headers carry different labels for several sections
+# ("DESIGN(term)"↔"DESIGN", "Modes (…)"↔"Toolbox modes", "CLI entry"↔"Main")
+# and the map lists "Transcript/Distill" with no banner of its own, so a
+# name gate would false-fail on the current file. The count gate is the
+# mechanical structural-sync check that stays green and still catches every
+# add/remove mutation.
+GP=0; GF=0
+ok(){ GP=$((GP+1)); printf '  PASS  %s\n' "$1"; }
+no(){ GF=$((GF+1)); printf '  FAIL  %s\n' "$1"; }
+
+SRC="$HERE/../scripts/summon.py"
+
+# docstring section names: from the first `Sections:` line to the next `"""`.
+doc_sections="$(awk '
+  !done && /Sections:/ { cap=1; sub(/.*Sections:[[:space:]]*/,"",$0); blob=blob $0 " "; next }
+  cap { if (/"""/) { done=1; cap=0; next } blob=blob $0 " " }
+  END { gsub(/·/,"\n",blob); n=split(blob,a,"\n");
+        for (i=1;i<=n;i++){ s=a[i]; sub(/^[[:space:]]+/,"",s); sub(/[[:space:]]+$/,"",s); sub(/\.$/,"",s); if (s!="") print s } }
+' "$SRC")"
+dc="$(printf '%s\n' "$doc_sections" | grep -c . || true)"
+
+# body banner sections: the `# ===…===` header pairs — the name line that
+# sits between each opening banner and its closing banner.
+ban_sections="$(awk '
+  /^# ={20,}$/ { saw=1; next }
+  saw && /^#  / { t=$0; sub(/^# +/,"",t); sub(/[[:space:]]+$/,"",t); print t }
+  { saw=0 }
+' "$SRC")"
+bc="$(printf '%s\n' "$ban_sections" | grep -c . || true)"
+
+# Expected counts — the docstring lists one more conceptual section than the
+# body has banners ("Transcript/Distill" is implemented inline, banner-less).
+# Bump EXPECT_DOC when the 'Sections:' map grows; EXPECT_BAN when a banner is
+# added or removed. Both move together whenever the file's outline changes.
+EXPECT_DOC=14
+EXPECT_BAN=13
+
+# forward direction (the map side): the declared section count is stable
+if [[ "$dc" -eq 0 ]]; then
+  no "section-map (docstring) EMPTY PARSE: 0 sections — 'Sections:' line missing or unparseable (expected $EXPECT_DOC)"
+elif [[ "$dc" -eq "$EXPECT_DOC" ]]; then
+  ok "section-map (docstring): $dc sections declared (expected $EXPECT_DOC)"
+else
+  no "section-map (docstring) DRIFT: parsed $dc, expected $EXPECT_DOC"
+fi
+# reverse direction (the body side): the banner section count is stable
+if [[ "$bc" -eq 0 ]]; then
+  no "section-map (body) EMPTY PARSE: 0 banners — banner format changed (expected $EXPECT_BAN)"
+elif [[ "$bc" -eq "$EXPECT_BAN" ]]; then
+  ok "section-map (body): $bc banner sections (expected $EXPECT_BAN)"
+else
+  no "section-map (body) DRIFT: parsed $bc, expected $EXPECT_BAN"
+fi
+
+echo "=== section-map drift gate: $GP passed, $GF failed ==="
+
+# Combine: the Python behavioural suite AND the shell section-map gate pass.
+[[ "$SUMMON_PY_RC" -eq 0 && "$GF" -eq 0 ]] || exit 1
+exit 0

+ 45 - 0
skills/svg-brand-tint-ops/tests/run.sh

@@ -86,5 +86,50 @@ fi
 kill "$SRV" >/dev/null 2>&1; trap - EXIT
 rm -f "$LOG" "$PORT_FILE" 2>/dev/null
 
+# --- section-map drift gate (assets/index.html: guard comment ↔ // === markers) ---
+# index.html is a deliberately single-file studio; its top <script> guard
+# comment lists the `// === NAME ===` banner sections so the file is
+# navigable. This gate keeps the guard list and the body markers in sync
+# bidirectionally and FAILS LOUDLY if either side parses to zero names — the
+# classic rot mode where a guard-comment/marker format change silently yields
+# an empty list and the check would otherwise vacuously pass.
+map_names="$(awk '
+  /Sections \(grep/ { cap=1; sub(/.*:[[:space:]]*/,"",$0); blob=blob $0 " "; if ($0 ~ /\*\//) cap=0; next }
+  cap { if ($0 ~ /\*\//) { cap=0; next } blob=blob $0 " " }
+  END { gsub(/·/,"\n",blob); n=split(blob,a,"\n");
+        for (i=1;i<=n;i++){ s=a[i]; sub(/^[[:space:]]+/,"",s); sub(/[[:space:]]+$/,"",s); if (s!="") print s } }
+' "$INDEX")"
+mark_names="$(grep -E '^// === .* ===$' "$INDEX" | sed -E 's|^// === (.*) ===$|\1|')"
+dc="$(printf '%s\n' "$map_names"  | grep -c . || true)"
+mc="$(printf '%s\n' "$mark_names" | grep -c . || true)"
+# empty-parse guard: either side unparseable is a hard fail (never a silent pass)
+if [[ "$dc" -gt 0 && "$mc" -gt 0 ]]; then
+  ok "section-map parses (guard=$dc names, body=$mc markers)"
+else
+  no "section-map EMPTY PARSE (guard=$dc, body=$mc) — guard comment or marker format changed"
+fi
+# forward: every guard-listed section has a matching // === marker
+fwd_miss=""
+while IFS= read -r n; do
+  [[ -z "$n" ]] && continue
+  grep -Fxq -- "$n" <<< "$mark_names" || fwd_miss="$fwd_miss $n"
+done <<< "$map_names"
+if [[ -z "$fwd_miss" ]]; then
+  ok "forward: every guard-listed section has a // === marker"
+else
+  no "forward: guard sections with no marker:${fwd_miss}"
+fi
+# reverse: every // === marker is present in the guard list
+rev_miss=""
+while IFS= read -r n; do
+  [[ -z "$n" ]] && continue
+  grep -Fxq -- "$n" <<< "$map_names" || rev_miss="$rev_miss $n"
+done <<< "$mark_names"
+if [[ -z "$rev_miss" ]]; then
+  ok "reverse: every // === marker is listed in the guard comment"
+else
+  no "reverse: body markers missing from guard:${rev_miss}"
+fi
+
 echo "=== $PASS passed, $FAIL failed ==="
 [[ "$FAIL" -eq 0 ]] || exit 1