Просмотр исходного кода

test(skills): python-fastapi/pytest/testing-ops suites + scaffold-api & coverage-check protocol backfill

Three offline, deterministic test suites (mktemp -d sandboxed; never write into
the repo or a real project), following the house run.sh style:

- python-fastapi-ops/tests/run.sh: contract (--help, exit codes, stream
  separation), generated-module structure, byte-identical idempotency, and the
  overwrite-safety story (scaffold-api emits to stdout and opens no file, so it
  is non-destructive; the only clobber is the caller's shell redirect, documented
  as a latent caller-side hazard, not 'fixed').
- python-pytest-ops/tests/run.sh: contract + generated conftest structure
  (base + --async/--db/--api), and overwrite safety (existing user conftest.py
  preserved on decline / closed-stdin; replaced only on explicit 'y').
- testing-ops/tests/run.sh: contract + threshold logic via the CM_COVERAGE_OVERRIDE
  offline seam (no real pytest), stream separation, and the exit-5 missing-dep
  path under a scrubbed PATH.

Script backfills to SKILL-RESOURCE-PROTOCOL:

- scaffold-api.sh: contract header, --help+EXAMPLES (exit 0), semantic exit
  codes (2 usage), stream separation (usage/errors to stderr). Generated stdout
  content is byte-identical pre/post (diff-verified: user 2839B, order 2900B).
- coverage-check.sh: contract header, --help+EXAMPLES, semantic exits
  (0 pass / 1 below-threshold / 2 usage / 5 missing-dep), stream separation
  (one-line verdict on stdout; banners + full pytest run on stderr), and an
  offline CM_COVERAGE_OVERRIDE test seam. Default pytest invocation preserved
  (--cov=src --cov-report=term-missing/html --cov-fail-under=80).

generate-conftest.sh: protocol header + early --help/-h (behavior-preserving).
Two unavoidable repairs required so the documented contract holds and bash -n
passes: (1) heredoc delimiter typos -- the --db and --api blocks opened with
<< 'DB_IMPORTS' / << 'API_IMPORTS' but both closed with a stray ASYNC_IMPORTS,
which broke bash -n and silently swallowed the rest of the script on --db/--api;
(2) a trailing 'exit 0' -- the script ended on '[[ -n "$API" ]] && echo',
returning 1 on every successful generation without --api, contradicting the
documented 'Exit: 0 ok'. Output text otherwise unchanged.

Verified: bash -n on all 6 files; all three suites green (28/34/31); tests/validate.sh
exit 0 (114/0/0); tests/check-exec-bits.sh clean; git ls-files -s shows 100755 for
all 6 touched files.
0xDarkMatter 6 дней назад
Родитель
Сommit
2b20105abf

+ 45 - 7
skills/python-fastapi-ops/scripts/scaffold-api.sh

@@ -1,17 +1,55 @@
 #!/usr/bin/env bash
-# Generate FastAPI endpoint boilerplate
+# Generate FastAPI endpoint boilerplate (Pydantic models + CRUD router) to stdout.
 #
-# Usage: scaffold-api.sh <resource_name>
-# Example: scaffold-api.sh user
+# Usage:   scaffold-api.sh <resource_name>
+# Input:   one positional arg — a singular resource name (e.g. "user", "order")
+# Output:  a complete FastAPI module (models + APIRouter CRUD endpoints) on stdout;
+#          redirect into a file to use it (the script emits to stdout and never
+#          opens a file itself, so it cannot clobber a user's project)
+# Stderr:  usage and error messages only
+# Exit:    0 ok, 2 usage (missing resource / unknown flag)
+#
+# Examples:
+#   scaffold-api.sh user > routers/user.py
+#   scaffold-api.sh order | tee routers/orders.py
+
+set -uo pipefail
+
+usage() {
+  cat <<'EOF'
+Usage: scaffold-api.sh <resource_name>
+
+Generate FastAPI endpoint boilerplate — Pydantic Create/Update/Response models
+plus an APIRouter with list/create/get/update/delete CRUD endpoints — printed to
+stdout. Redirect into a module file to use it.
 
-set -euo pipefail
+Arguments:
+  resource_name   a singular resource name, e.g. "user" or "order"
+
+Exit codes: 0 ok, 2 usage (missing resource / unknown flag).
+
+Examples:
+  scaffold-api.sh user > routers/user.py
+  scaffold-api.sh order | tee routers/orders.py
+EOF
+}
+
+# Flags first: --help short-circuits; any unknown flag is a hard usage error.
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    -h|--help) usage; exit 0 ;;
+    --) shift; break ;;
+    -*) echo "scaffold-api.sh: unknown option: $1" >&2; usage >&2; exit 2 ;;
+    *) break ;;
+  esac
+done
 
 RESOURCE="${1:-}"
 
 if [[ -z "$RESOURCE" ]]; then
-    echo "Usage: scaffold-api.sh <resource_name>"
-    echo "Example: scaffold-api.sh user"
-    exit 1
+  echo "scaffold-api.sh: resource name required" >&2
+  usage >&2
+  exit 2
 fi
 
 # Convert to different cases

+ 94 - 0
skills/python-fastapi-ops/tests/run.sh

@@ -0,0 +1,94 @@
+#!/usr/bin/env bash
+# Self-test for python-fastapi-ops — fully offline, deterministic, Linux-safe.
+#
+# scaffold-api.sh emits a FastAPI module to stdout; it never opens a file for
+# writing, so it cannot clobber a user's project. Every run here is redirected
+# into a mktemp -d sandbox — nothing is ever written into the repo or a real
+# project. Asserts the protocol contract (--help, exit codes, stream
+# separation), the generated module's structure, byte-identical idempotency,
+# and that the script owns no overwrite hazard (the only clobber is the
+# caller's shell `>` redirect — a latent caller-side hazard, documented here,
+# NOT "fixed" in the script per its contract).
+#
+# Usage:   bash tests/run.sh
+# Exit:    0 all pass, 1 one or more failures
+
+set -uo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SKILL="$(dirname "$HERE")"
+V="$SKILL/scripts/scaffold-api.sh"
+
+SB="$(mktemp -d)"; trap 'rm -rf "$SB"' EXIT
+PASS=0; FAIL=0
+ok() { PASS=$((PASS+1)); printf '  PASS  %s\n' "$1"; }
+no() { FAIL=$((FAIL+1)); printf '  FAIL  %s\n' "$1"; }
+expect_exit() { [[ "$2" == "$3" ]] && ok "$1 (exit $3)" || no "$1 (want $2 got $3)"; }
+expect_has()  { case "$3" in *"$2"*) ok "$1";; *) no "$1 (missing '$2')";; esac; }
+
+echo "=== python-fastapi-ops self-test ==="
+
+# ── contract ──────────────────────────────────────────────────────────────────
+echo "-- contract --"
+bash -n "$V" 2>/dev/null && ok "bash -n scaffold-api.sh" || no "bash -n scaffold-api.sh"
+bash "$V" --help >/dev/null 2>&1; expect_exit "--help exits 0" 0 $?
+bash "$V" -h     >/dev/null 2>&1; expect_exit "-h exits 0" 0 $?
+out="$(bash "$V" --help 2>/dev/null)"
+expect_has "--help has Examples" "xamples" "$out"
+expect_has "--help documents exit 2" "2" "$out"
+bash "$V"          >/dev/null 2>&1; expect_exit "missing resource -> 2" 2 $?
+bash "$V" --bogus  >/dev/null 2>&1; expect_exit "unknown flag -> 2" 2 $?
+
+# usage/errors must go to stderr, never polluting the stdout data stream
+err="$(bash "$V" 2>&1 >/dev/null)"
+expect_has "missing-resource usage on stderr" "Usage:" "$err"
+noso="$(bash "$V" --bogus 2>/dev/null)"
+[[ -z "$noso" ]] && ok "unknown-flag writes nothing to stdout" || no "unknown-flag leaked to stdout"
+
+# ── generated module structure ────────────────────────────────────────────────
+echo "-- generated module --"
+bash "$V" user >"$SB/user.py" 2>"$SB/user.err"; expect_exit "generate user -> 0" 0 $?
+[[ -s "$SB/user.py" ]] && ok "module written to redirected stdout" || no "module not written"
+[[ ! -s "$SB/user.err" ]] && ok "happy path is silent on stderr" || no "happy path wrote to stderr"
+out="$(cat "$SB/user.py")"
+expect_has "has Pydantic Create model"  "class UserCreate(BaseModel):" "$out"
+expect_has "has Pydantic Update model"  "class UserUpdate(BaseModel):" "$out"
+expect_has "has Pydantic Response model" "class UserResponse(BaseModel):" "$out"
+expect_has "has APIRouter with prefix"  'router = APIRouter(prefix="/users"' "$out"
+expect_has "has create endpoint"  "async def create_user" "$out"
+expect_has "has list endpoint"    "async def list_users" "$out"
+expect_has "has get endpoint"     "async def get_user" "$out"
+expect_has "has update endpoint"  "async def update_user" "$out"
+expect_has "has delete endpoint"  "async def delete_user" "$out"
+expect_has "resource name in Create docstring" "Create user request" "$out"
+
+# title-casing + pluralization for mixed-case input
+bash "$V" Order >"$SB/order.py" 2>/dev/null; expect_exit "generate Order -> 0" 0 $?
+out="$(cat "$SB/order.py")"
+expect_has "title-cases Order"        "class OrderCreate(BaseModel):" "$out"
+expect_has "pluralizes to orders"     'prefix="/orders"' "$out"
+expect_has "orders create endpoint"   "async def create_order" "$out"
+
+# ── idempotency: re-running produces byte-identical output ───────────────────
+echo "-- idempotency --"
+bash "$V" user >"$SB/a.py" 2>/dev/null
+bash "$V" user >"$SB/b.py" 2>/dev/null
+if cmp -s "$SB/a.py" "$SB/b.py"; then ok "re-run produces byte-identical output"; else no "re-run output differs"; fi
+
+# ── overwrite safety ─────────────────────────────────────────────────────────
+echo "-- overwrite safety --"
+# The script emits to stdout only and never opens a file, so it is
+# non-destructive by construction: running it leaves the cwd untouched. There
+# is no --force/--no-clobber flag because the script has no file target to
+# guard — the sole clobber path is the caller's own shell redirect (`>`),
+# which the script cannot see. That caller-side hazard is documented here,
+# not "fixed".
+mkdir -p "$SB/cwd-check"
+( cd "$SB/cwd-check" && bash "$V" user >/dev/null 2>&1 )
+n="$(find "$SB/cwd-check" -type f | wc -l | tr -d ' ')"
+[[ "$n" == "0" ]] && ok "script creates no files in cwd (non-destructive)" || no "script wrote files into cwd ($n)"
+
+echo ""
+echo "=== $PASS passed, $FAIL failed ==="
+[[ "$FAIL" -eq 0 ]] || exit 1
+exit 0

+ 43 - 4
skills/python-pytest-ops/scripts/generate-conftest.sh

@@ -1,9 +1,46 @@
 #!/bin/bash
-# Generate conftest.py boilerplate
-# Usage: ./generate-conftest.sh [--async] [--db] [--api]
+# Generate a pytest conftest.py with optional async/db/api fixture blocks.
+#
+# Usage:   generate-conftest.sh [--async] [--db] [--api]
+# Input:   optional flags selecting fixture blocks; writes ./tests/conftest.py
+# Output:  the generated conftest.py file plus a one-line summary on stdout
+# Stderr:  none (the overwrite confirmation prompt is read from the tty)
+# Exit:    0 ok (also --help, and when an existing file is kept at the prompt)
+#
+# Examples:
+#   generate-conftest.sh
+#   generate-conftest.sh --async --db
+#   generate-conftest.sh --api
 
 set -e
 
+# --help / -h: print usage to stdout and exit 0 before any side effects (so the
+# prompt never fires and no file is touched). Behavior-preserving for all other
+# invocations — only -h/--help short-circuits here.
+for _arg in "$@"; do
+  case "$_arg" in
+    -h|--help)
+      cat <<'EOF'
+Usage: generate-conftest.sh [--async] [--db] [--api]
+
+Generate ./tests/conftest.py with optional fixture blocks. If a conftest.py
+already exists, it prompts for confirmation (default: keep the existing file).
+
+Options:
+  --async   include async fixtures (event_loop, async_client)
+  --db      include database fixtures (db_engine, db_session)
+  --api     include API test fixtures (app, client, authenticated_client)
+
+Examples:
+  generate-conftest.sh
+  generate-conftest.sh --async --db
+  generate-conftest.sh --api
+EOF
+      exit 0
+      ;;
+  esac
+done
+
 OUTPUT="tests/conftest.py"
 
 # Check if tests directory exists
@@ -67,7 +104,7 @@ if [[ -n "$DB" ]]; then
     cat >> "$OUTPUT" << 'DB_IMPORTS'
 from sqlalchemy import create_engine
 from sqlalchemy.orm import sessionmaker
-ASYNC_IMPORTS
+DB_IMPORTS
 fi
 
 # Add API imports if needed
@@ -75,7 +112,7 @@ if [[ -n "$API" ]]; then
     cat >> "$OUTPUT" << 'API_IMPORTS'
 from fastapi.testclient import TestClient
 # or: from flask.testing import FlaskClient
-ASYNC_IMPORTS
+API_IMPORTS
 fi
 
 # Add base fixtures
@@ -227,3 +264,5 @@ echo "Options used:"
 [[ -n "$ASYNC" ]] && echo "  --async: Async fixtures included"
 [[ -n "$DB" ]] && echo "  --db: Database fixtures included"
 [[ -n "$API" ]] && echo "  --api: API fixtures included"
+
+exit 0

+ 105 - 0
skills/python-pytest-ops/tests/run.sh

@@ -0,0 +1,105 @@
+#!/usr/bin/env bash
+# Self-test for python-pytest-ops — fully offline, deterministic, Linux-safe.
+#
+# generate-conftest.sh writes ./tests/conftest.py relative to the cwd and, when
+# one already exists, prompts interactively (defaulting to KEEP the file). Every
+# run executes inside a mktemp -d sandbox so nothing is ever written into the
+# repo or a real project. Asserts the --help contract, the generated conftest's
+# fixture blocks (base + --async/--db/--api), and the overwrite-safety guarantee:
+# an existing user conftest.py is NOT clobbered unless the user answers the
+# prompt with "y" (the documented mechanism — fed "n" here keeps the file).
+#
+# Usage:   bash tests/run.sh
+# Exit:    0 all pass, 1 one or more failures
+
+set -uo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SKILL="$(dirname "$HERE")"
+V="$SKILL/scripts/generate-conftest.sh"
+
+SB="$(mktemp -d)"; trap 'rm -rf "$SB"' EXIT
+PASS=0; FAIL=0
+ok() { PASS=$((PASS+1)); printf '  PASS  %s\n' "$1"; }
+no() { FAIL=$((FAIL+1)); printf '  FAIL  %s\n' "$1"; }
+expect_exit() { [[ "$2" == "$3" ]] && ok "$1 (exit $3)" || no "$1 (want $2 got $3)"; }
+expect_has()  { case "$3" in *"$2"*) ok "$1";; *) no "$1 (missing '$2')";; esac; }
+
+echo "=== python-pytest-ops self-test ==="
+
+# ── contract ──────────────────────────────────────────────────────────────────
+echo "-- contract --"
+bash -n "$V" 2>/dev/null && ok "bash -n generate-conftest.sh" || no "bash -n generate-conftest.sh"
+bash "$V" --help >/dev/null 2>&1; expect_exit "--help exits 0" 0 $?
+bash "$V" -h     >/dev/null 2>&1; expect_exit "-h exits 0" 0 $?
+out="$(bash "$V" --help 2>/dev/null)"
+expect_has "--help has Examples" "xamples" "$out"
+expect_has "--help lists --async" "--async" "$out"
+expect_has "--help lists --db"    "--db" "$out"
+expect_has "--help lists --api"   "--api" "$out"
+# --help must short-circuit before any side effect (no file created even though
+# the cwd has no tests/ dir)
+mkdir -p "$SB/help-cwd"
+( cd "$SB/help-cwd" && bash "$V" --help >/dev/null 2>&1 )
+[[ ! -e "$SB/help-cwd/tests" ]] && ok "--help creates no files" || no "--help created files"
+
+# ── fresh generation ──────────────────────────────────────────────────────────
+echo "-- fresh generation --"
+mkdir -p "$SB/fresh"
+( cd "$SB/fresh" && bash "$V" </dev/null >"$SB/fresh.out" 2>"$SB/fresh.err" ); expect_exit "fresh generate -> 0" 0 $?
+[[ -f "$SB/fresh/tests/conftest.py" ]] && ok "creates tests/conftest.py" || no "conftest.py not created"
+out="$(cat "$SB/fresh/tests/conftest.py")"
+expect_has "has module docstring"      "Pytest configuration and fixtures" "$out"
+expect_has "imports pytest"            "import pytest" "$out"
+expect_has "registers slow marker"     '"slow: marks tests as slow"' "$out"
+expect_has "has sample_data fixture"   "def sample_data():" "$out"
+expect_has "has temp_file fixture"     "def temp_file(tmp_path):" "$out"
+expect_has "has pytest_addoption"      "def pytest_addoption(parser):" "$out"
+expect_has "summary on stdout"         "Generated tests/conftest.py" "$(cat "$SB/fresh.out")"
+
+# ── fixture-block flags ───────────────────────────────────────────────────────
+echo "-- fixture blocks --"
+mkdir -p "$SB/all"
+( cd "$SB/all" && bash "$V" --async --db --api </dev/null >/dev/null 2>&1 ); expect_exit "all flags -> 0" 0 $?
+out="$(cat "$SB/all/tests/conftest.py")"
+expect_has "--async adds event_loop"    "def event_loop():" "$out"
+expect_has "--async adds async_client"  "async def async_client():" "$out"
+expect_has "--db adds db_engine"        "def db_engine():" "$out"
+expect_has "--db adds db_session"       "def db_session(db_engine):" "$out"
+expect_has "--api adds client fixture"  "def client(app):" "$out"
+expect_has "--api adds authed client"   "def authenticated_client(client):" "$out"
+
+# base generation must NOT include the optional blocks
+! grep -q "def event_loop" "$SB/fresh/tests/conftest.py" && ok "base omits async blocks" || no "base included async blocks"
+! grep -q "def db_engine"  "$SB/fresh/tests/conftest.py" && ok "base omits db blocks"    || no "base included db blocks"
+! grep -q "def client"     "$SB/fresh/tests/conftest.py" && ok "base omits api blocks"   || no "base included api blocks"
+
+# ── overwrite safety ─────────────────────────────────────────────────────────
+echo "-- overwrite safety --"
+# Decline ("n", the documented default) MUST preserve an existing user conftest.
+mkdir -p "$SB/ow/tests"
+printf '# USER-SENTINEL: do not overwrite me\nimport user_thing\n' >"$SB/ow/tests/conftest.py"
+( cd "$SB/ow" && printf 'n\n' | bash "$V" >/dev/null 2>&1 ); expect_exit "decline overwrite -> 0" 0 $?
+out="$(cat "$SB/ow/tests/conftest.py")"
+expect_has "user conftest preserved (decline)" "USER-SENTINEL" "$out"
+expect_has "user import preserved (decline)"   "import user_thing" "$out"
+
+# Confirming ("y") DOES replace the file — that is the documented overwrite path.
+mkdir -p "$SB/ow2/tests"
+printf '# USER-SENTINEL-2\n' >"$SB/ow2/tests/conftest.py"
+( cd "$SB/ow2" && printf 'y\n' | bash "$V" >/dev/null 2>&1 ); expect_exit "confirm overwrite -> 0" 0 $?
+out="$(cat "$SB/ow2/tests/conftest.py")"
+! grep -q "USER-SENTINEL-2" "$SB/ow2/tests/conftest.py" && ok "confirm 'y' replaces user conftest" || no "confirm 'y' did not replace"
+expect_has "new conftest has pytest import" "import pytest" "$out"
+
+# Non-interactive (closed stdin, no answer) also keeps the existing file — the
+# safe default when there is no TTY to answer the prompt.
+mkdir -p "$SB/ow3/tests"
+printf '# USER-SENTINEL-3\n' >"$SB/ow3/tests/conftest.py"
+( cd "$SB/ow3" && bash "$V" </dev/null >/dev/null 2>&1 )
+expect_has "closed-stdin keeps user conftest" "USER-SENTINEL-3" "$(cat "$SB/ow3/tests/conftest.py")"
+
+echo ""
+echo "=== $PASS passed, $FAIL failed ==="
+[[ "$FAIL" -eq 0 ]] || exit 1
+exit 0

+ 96 - 27
skills/testing-ops/scripts/coverage-check.sh

@@ -1,38 +1,107 @@
-#!/bin/bash
-# Run tests with coverage and fail if below threshold
-# Usage: ./coverage-check.sh [--threshold 80] [pytest-args...]
+#!/usr/bin/env bash
+# Run pytest with coverage and exit non-zero when coverage is below a threshold.
+#
+# Usage:   coverage-check.sh [--threshold N] [--cov TARGET] [pytest-args...]
+# Input:   a pytest project in the current directory; OR CM_COVERAGE_OVERRIDE=PCT
+#          to judge a given percentage offline (test seam — skips pytest entirely)
+# Output:  one verdict line on stdout, e.g. "coverage-check<TAB>pass<TAB>90<TAB>80"
+# Stderr:  status banners and the full pytest run (progress, coverage report, errors)
+# Exit:    0 pass, 1 below threshold, 2 usage, 5 pytest missing
+#
+# Examples:
+#   coverage-check.sh
+#   coverage-check.sh --threshold 90 --cov mypkg
+#   CM_COVERAGE_OVERRIDE=72 coverage-check.sh --threshold 80   # offline test mode
 
-set -e
+set -uo pipefail
 
 THRESHOLD=80
-PYTEST_ARGS=""
+COV_TARGET="src"
+PYTEST_ARGS=()
+OVERRIDE="${CM_COVERAGE_OVERRIDE:-}"
+
+usage() {
+  cat <<'EOF'
+Usage: coverage-check.sh [--threshold N] [--cov TARGET] [pytest-args...]
+
+Run pytest with coverage; exit non-zero when coverage is below --threshold.
+The verdict is one line on stdout; pytest output and status banners go to stderr.
+
+Options:
+  --threshold N   minimum coverage percent (default 80)
+  --cov TARGET    package to measure (default src)
+
+Offline test seam:
+  CM_COVERAGE_OVERRIDE=PCT   skip pytest, judge this percentage instead.
+
+Exit codes: 0 pass, 1 below threshold, 2 usage, 5 pytest missing.
+
+Examples:
+  coverage-check.sh
+  coverage-check.sh --threshold 90 --cov mypkg
+  CM_COVERAGE_OVERRIDE=72 coverage-check.sh --threshold 80
+EOF
+}
 
-# Parse arguments
 while [[ $# -gt 0 ]]; do
-    case $1 in
-        --threshold)
-            THRESHOLD="$2"
-            shift 2
-            ;;
-        *)
-            PYTEST_ARGS="$PYTEST_ARGS $1"
-            shift
-            ;;
-    esac
+  case "$1" in
+    -h|--help) usage; exit 0 ;;
+    --threshold)
+      [[ $# -ge 2 ]] || { echo "coverage-check.sh: --threshold needs a value" >&2; exit 2; }
+      THRESHOLD="$2"; shift 2 ;;
+    --threshold=*) THRESHOLD="${1#--threshold=}"; shift ;;
+    --cov)
+      [[ $# -ge 2 ]] || { echo "coverage-check.sh: --cov needs a value" >&2; exit 2; }
+      COV_TARGET="$2"; shift 2 ;;
+    --cov=*) COV_TARGET="${1#--cov=}"; shift ;;
+    --) shift; while [[ $# -gt 0 ]]; do PYTEST_ARGS+=("$1"); shift; done ;;
+    -*) echo "coverage-check.sh: unknown option: $1" >&2; usage >&2; exit 2 ;;
+    *) PYTEST_ARGS+=("$1"); shift ;;
+  esac
 done
 
-echo "=== Running tests with coverage ==="
-echo "Minimum coverage threshold: ${THRESHOLD}%"
-echo ""
+# threshold must be a number
+if ! [[ "$THRESHOLD" =~ ^[0-9]+([.][0-9]+)?$ ]]; then
+  echo "coverage-check.sh: --threshold must be a number, got '$THRESHOLD'" >&2
+  exit 2
+fi
+
+# ge PCT THRESHOLD -> returns 0 if PCT >= THRESHOLD (float-safe via awk)
+ge() { awk -v a="$1" -v b="$2" 'BEGIN { exit !(a+0 >= b+0) }'; }
+
+# Offline test seam: judge a supplied percentage without running pytest. This is
+# what lets the threshold logic be exercised offline, deterministically, with no
+# slow suite and no network — the same pattern check-ytdlp-version.sh uses.
+if [[ -n "$OVERRIDE" ]]; then
+  if ! [[ "$OVERRIDE" =~ ^[0-9]+([.][0-9]+)?$ ]]; then
+    echo "coverage-check.sh: CM_COVERAGE_OVERRIDE must be a number, got '$OVERRIDE'" >&2
+    exit 2
+  fi
+  if ge "$OVERRIDE" "$THRESHOLD"; then
+    printf 'coverage-check\tpass\t%s\t%s\n' "$OVERRIDE" "$THRESHOLD"; exit 0
+  else
+    printf 'coverage-check\tfail\t%s\t%s\n' "$OVERRIDE" "$THRESHOLD"; exit 1
+  fi
+fi
 
-# Run pytest with coverage
+# Live path: run pytest with coverage. All of its output (progress + coverage
+# report) is data for a human, not for this script's caller, so it goes to
+# stderr; the one-line verdict on stdout is the machine-readable product.
+command -v pytest >/dev/null 2>&1 || {
+  echo "coverage-check.sh: pytest not found (pip install pytest pytest-cov)" >&2
+  exit 5
+}
+
+printf '=== Running tests with coverage (threshold %s%%) ===\n' "$THRESHOLD" >&2
 pytest \
-    --cov=src \
+    --cov="$COV_TARGET" \
     --cov-report=term-missing \
     --cov-report=html \
-    --cov-fail-under=${THRESHOLD} \
-    ${PYTEST_ARGS}
-
-echo ""
-echo "=== Coverage report generated ==="
-echo "HTML report: htmlcov/index.html"
+    --cov-fail-under="$THRESHOLD" \
+    "${PYTEST_ARGS[@]}" >&2
+rc=$?
+case "$rc" in
+  0) printf 'coverage-check\tpass\t-\t%s\n' "$THRESHOLD"; exit 0 ;;
+  2) printf 'coverage-check\tfail\t-\t%s\n' "$THRESHOLD"; exit 1 ;;  # pytest-cov below-threshold signal
+  *) printf 'coverage-check.sh: pytest exited %s\n' "$rc" >&2; exit "$rc" ;;
+esac

+ 91 - 0
skills/testing-ops/tests/run.sh

@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+# Self-test for testing-ops — fully offline, deterministic, Linux-safe.
+#
+# coverage-check.sh gates a project on a coverage threshold. To exercise its
+# threshold logic WITHOUT running a slow real pytest suite, the script exposes
+# an offline test seam (CM_COVERAGE_OVERRIDE=PCT) that substitutes a given
+# percentage for the live measurement — the same pattern check-ytdlp-version.sh
+# uses for its installed/latest versions. Asserts the --help contract, semantic
+# exit codes (0 pass / 1 below / 2 usage / 5 missing-dep), and stream separation
+# (the verdict on stdout; banners + pytest on stderr). Never invokes real pytest.
+#
+# Usage:   bash tests/run.sh
+# Exit:    0 all pass, 1 one or more failures
+
+set -uo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SKILL="$(dirname "$HERE")"
+V="$SKILL/scripts/coverage-check.sh"
+
+SB="$(mktemp -d)"; trap 'rm -rf "$SB"' EXIT
+PASS=0; FAIL=0
+ok() { PASS=$((PASS+1)); printf '  PASS  %s\n' "$1"; }
+no() { FAIL=$((FAIL+1)); printf '  FAIL  %s\n' "$1"; }
+expect_exit() { [[ "$2" == "$3" ]] && ok "$1 (exit $3)" || no "$1 (want $2 got $3)"; }
+expect_has()  { case "$3" in *"$2"*) ok "$1";; *) no "$1 (missing '$2')";; esac; }
+
+echo "=== testing-ops self-test ==="
+
+# ── contract ──────────────────────────────────────────────────────────────────
+echo "-- contract --"
+bash -n "$V" 2>/dev/null && ok "bash -n coverage-check.sh" || no "bash -n coverage-check.sh"
+bash "$V" --help >/dev/null 2>&1; expect_exit "--help exits 0" 0 $?
+bash "$V" -h     >/dev/null 2>&1; expect_exit "-h exits 0" 0 $?
+out="$(bash "$V" --help 2>/dev/null)"
+expect_has "--help has Examples"          "xamples" "$out"
+expect_has "--help documents exit 1 (below)" "below threshold" "$out"
+expect_has "--help documents exit 2 (usage)" "usage" "$out"
+expect_has "--help documents exit 5 (missing-dep)" "pytest missing" "$out"
+expect_has "--help names the test seam"   "CM_COVERAGE_OVERRIDE" "$out"
+bash "$V" --bogus            >/dev/null 2>&1; expect_exit "unknown flag -> 2" 2 $?
+bash "$V" --threshold        >/dev/null 2>&1; expect_exit "--threshold needs value -> 2" 2 $?
+bash "$V" --threshold notnum >/dev/null 2>&1; expect_exit "--threshold non-numeric -> 2" 2 $?
+CM_COVERAGE_OVERRIDE=bogus bash "$V" --threshold 80 >/dev/null 2>&1; expect_exit "bad override -> 2" 2 $?
+
+# ── threshold logic (offline seam; no real pytest) ───────────────────────────
+echo "-- threshold logic (seamed) --"
+CM_COVERAGE_OVERRIDE=90 bash "$V" --threshold 80 >/dev/null 2>&1; expect_exit "90>=80 -> pass 0" 0 $?
+CM_COVERAGE_OVERRIDE=80 bash "$V" --threshold 80 >/dev/null 2>&1; expect_exit "80>=80 boundary -> pass 0" 0 $?
+CM_COVERAGE_OVERRIDE=79 bash "$V" --threshold 80 >/dev/null 2>&1; expect_exit "79<80 -> below 1" 1 $?
+CM_COVERAGE_OVERRIDE=72 bash "$V" --threshold 80 >/dev/null 2>&1; expect_exit "72<80 -> below 1" 1 $?
+CM_COVERAGE_OVERRIDE=100 bash "$V" --threshold 99.5 >/dev/null 2>&1; expect_exit "100>=99.5 float -> pass 0" 0 $?
+CM_COVERAGE_OVERRIDE=99.4 bash "$V" --threshold 99.5 >/dev/null 2>&1; expect_exit "99.4<99.5 float -> below 1" 1 $?
+
+# default threshold is 80 when --threshold is omitted
+CM_COVERAGE_OVERRIDE=79 bash "$V" >/dev/null 2>&1; expect_exit "default threshold 80: 79 -> below 1" 1 $?
+CM_COVERAGE_OVERRIDE=81 bash "$V" >/dev/null 2>&1; expect_exit "default threshold 80: 81 -> pass 0" 0 $?
+
+# ── stream separation (verdict on stdout; no banner leaks) ───────────────────
+echo "-- stream separation --"
+out="$(CM_COVERAGE_OVERRIDE=90 bash "$V" --threshold 80 2>/dev/null)"; rc=$?
+expect_exit "seamed pass exit 0" 0 "$rc"
+expect_has "stdout verdict is pass"       "pass" "$out"
+expect_has "stdout verdict carries threshold" "80" "$out"
+case "$out" in *$'\n'*) no "stdout has more than one line";; *) ok "stdout is a single verdict line";; esac
+case "$out" in *"==="*)  no "banner leaked onto stdout";;       *) ok "no banner on stdout";; esac
+
+out="$(CM_COVERAGE_OVERRIDE=72 bash "$V" --threshold 80 2>/dev/null)"; rc=$?
+expect_exit "seamed fail exit 1" 1 "$rc"
+expect_has "stdout verdict is fail" "fail" "$out"
+case "$out" in *"==="*) no "banner leaked onto stdout (fail)";; *) ok "no banner on stdout (fail)";; esac
+
+# ── missing-dep path: no seam AND no pytest -> exit 5 with install hint ───────
+echo "-- missing-dep --"
+# Scrub PATH so a host-installed pytest is not resolvable, keeping the suite
+# offline and deterministic regardless of host tooling (bash itself stays
+# resolvable under /usr/bin:/bin). If pytest somehow survives the scrub, SKIP
+# rather than ever launching a real suite.
+if PATH=/usr/bin:/bin command -v pytest >/dev/null 2>&1; then
+  echo "  SKIP  missing-dep exit 5 (pytest resolvable even under scrubbed PATH)"
+else
+  out="$(PATH=/usr/bin:/bin bash "$V" 2>&1)"; rc=$?
+  expect_exit "no pytest, no seam -> 5" 5 "$rc"
+  expect_has "names pytest install hint" "pytest" "$out"
+  case "$out" in *"pip install pytest"*) ok "hint suggests install";; *) no "hint missing install suggestion";; esac
+fi
+
+echo ""
+echo "=== $PASS passed, $FAIL failed ==="
+[[ "$FAIL" -eq 0 ]] || exit 1
+exit 0