Ver Fonte

feat(ci): close pull requests that drop a template section (#6881)

* feat(ci): close pull requests that drop a template section

Contributors sometimes open a pull request whose body has replaced or
dropped sections of .github/pull_request_template.md rather than
filling them in. Add a workflow that checks the body against the
template's section headings on open and reopen, and closes the pull
request with an explanation when a section is missing, so this is
caught immediately rather than discovered by a maintainer later.

Bot authors and pull requests labelled template-check-overridden are
skipped. The check only requires each section heading to still be
present; it deliberately does not judge the content underneath, to
keep the false-positive risk of closing a legitimate contribution low.

Fixes: external-secrets/external-secrets#6879
Signed-off-by: Alexander Chernov <alexander@chernov.it>

* fix(ci): tighten fence and heading matching in the template check

Two issues from automated review on external-secrets/external-secrets#6881:
stripFences required the closing fence to exactly match the opening one,
so a longer closing fence (opened with three backticks, closed with four)
was not stripped, letting a heading-like line inside it leak through.
Fixed by matching the fence character with an independent minimum length
on each side instead of a whole-delimiter backreference.

missingHeadings matched a template heading anywhere inside a body heading,
so "Reformat" satisfied the required "Format" section. Tightened to an
exact normalised match or a word-boundary prefix, which still allows an
elaborated heading such as "Related Issue / Ticket".

Refs: external-secrets/external-secrets#6879
Signed-off-by: Alexander Chernov <alexander@chernov.it>

* feat(ci): check checklist and AI disclosure content, not headings

The issue reporter clarified in a maintainer discussion what they actually
want checked: not that every template section heading survives, but that
the Checklist keeps every item's text (regardless of tick state) and that
the AI Assistance disclosure is genuinely answered, with the four detail
fields filled in when the answer is Yes. Problem Statement, Related Issue,
Proposed Changes and Format are no longer checked at all.

Replaces the heading-diff mechanism with two targeted checks: extractSection
locates a section's body between headings, extractChecklistItems finds each
checkbox item's label text, and fieldValue/checkAiDisclosure read the
disclosure's labelled fields, including the case where a field's answer sits
on the line below its label rather than after the colon.

Two independent review passes on this rewrite found bugs before it went out:

- Every extraction regex anchored end-of-line without the multiline flag,
  relying on `.`/`.*` stopping at `\n`. JavaScript's `.` does not match `\r`,
  so a CRLF body (what the GitHub web editor produces) made every section
  read as empty and inverted the whole check: a fully conformant pull
  request got closed with fabricated missing-section claims. Fixed with one
  line-ending normalisation at the top of checkConformance.
- The unedited-placeholder guard was a single literal-string check, so
  emphasising or striking one option instead of deleting it, or reordering
  it, slipped past as an answer. Replaced with a shape check: an answer
  containing both a yes-token and a no-token is ambiguous regardless of
  punctuation or order.
- Added "None"/"N/A"/"Not applicable" as recognised No-answers, and numbered
  list markers alongside bulleted ones for checklist items.

Also adds a pull request number cutoff: anything numbered at or below the
highest pull request in the repository on the day this landed is skipped
even on reopen, so nothing that predates the check is judged against a rule
it never saw.

Refs: external-secrets/external-secrets#6879
Signed-off-by: Alexander Chernov <alexander@chernov.it>

* fix(ci): track fence state by line instead of one regex

CodeRabbit found a Major issue that survived the earlier fence fix: two
independent {3,} quantifiers meant a closer shorter than its opener could
still end the fence, since nothing compared the two lengths, and an
unclosed fence was left untouched instead of running to end of document.
Either case exposes a heading-like line inside what should still be fenced
content, which can truncate a real section (Checklist or AI disclosure)
right where the fake heading appears.

A regex backreference can't parametrise "at least as many repeats as this
specific opener", so this replaces it with a line-by-line scan that
remembers the opening fence's character and length and only accepts a
closer of the same character with at least that length. An unclosed fence
now runs to end of document, matching how GitHub renders one. Added
regressions for both cases.

Also exports AI_ASSISTANCE_LINE_LABEL and AI_DETAIL_FIELDS so the
template-drift test imports the real constants instead of asserting
against its own separately hand-typed copy, which could drift from what
the check actually enforces without the test ever noticing (CodeRabbit
nit).

Refs: external-secrets/external-secrets#6879
Signed-off-by: Alexander Chernov <alexander@chernov.it>

---------

Signed-off-by: Alexander Chernov <alexander@chernov.it>
Co-authored-by: Jean-Philippe Evrard <jean-philippe.evrard+rochepub@external.roche.com>
Alexander Chernov há 16 horas atrás
pai
commit
e70fd917de

+ 556 - 0
.github/scripts/pr-template-conformance-test.js

@@ -0,0 +1,556 @@
+/**
+ * Tests for pr-template-conformance.js.
+ *
+ * Reads the real .github/pull_request_template.md rather than a fixture
+ * copy, so the tests cannot drift from what the workflow actually checks
+ * against.
+ *
+ * Run with: node .github/scripts/pr-template-conformance-test.js
+ */
+
+import assert from 'node:assert/strict';
+import test from 'node:test';
+import { readFileSync } from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import run, {
+  isBot, OVERRIDE_LABEL, CUTOFF_PR_NUMBER, AI_ASSISTANCE_LINE_LABEL,
+  AI_DETAIL_FIELDS, normalise, extractSection, extractChecklistItems,
+  missingChecklistItems, checkAiDisclosure, checkConformance, closeMessage,
+} from './pr-template-conformance.js';
+
+const SCRIPTS_DIR = path.dirname(fileURLToPath(import.meta.url));
+const REAL_TEMPLATE = readFileSync(path.join(SCRIPTS_DIR, '..', 'pull_request_template.md'), 'utf8');
+
+function fakeCore() {
+  const infos = [];
+  const failures = [];
+  return {
+    infos,
+    failures,
+    info: (m) => infos.push(m),
+    warning: () => {},
+    setFailed: (m) => failures.push(m),
+  };
+}
+
+// Captures what the workflow would have written, without touching GitHub.
+function fakeGithub() {
+  const comments = [];
+  const closed = [];
+  return {
+    comments,
+    closed,
+    rest: {
+      issues: {
+        createComment: async (args) => { comments.push(args); },
+      },
+      pulls: {
+        update: async (args) => { closed.push(args); },
+      },
+    },
+  };
+}
+
+function pr(overrides = {}) {
+  return {
+    number: CUTOFF_PR_NUMBER + 1,
+    user: { login: 'contributor' },
+    labels: [],
+    body: REAL_TEMPLATE,
+    ...overrides,
+  };
+}
+
+const context = (payload) => ({ repo: { owner: 'external-secrets', repo: 'external-secrets' }, payload });
+
+// A filled-in Checklist section, matching every real item, for tests that
+// only care about the AI Assistance disclosure.
+const FILLED_CHECKLIST = extractSection(REAL_TEMPLATE, 'Checklist')
+  .replace(/\[ \]/g, '[x]');
+
+function withSections({ checklist = FILLED_CHECKLIST, aiDisclosure } = {}) {
+  const base = REAL_TEMPLATE
+    .replace(/## Checklist[\s\S]*$/, `## Checklist\n${checklist}\n`);
+  if (aiDisclosure === undefined) return base;
+  return base.replace(
+    /## AI Assistance disclosure[\s\S]*?(?=\n## )/,
+    `## AI Assistance disclosure\n\n${aiDisclosure}\n\n`,
+  );
+}
+
+// ---------------------------------------------------------------------------
+// normalise
+// ---------------------------------------------------------------------------
+
+test('normalise strips markup, case and punctuation, collapses whitespace', () => {
+  assert.equal(normalise('**AI Assistance disclosure!**'), 'ai assistance disclosure');
+  assert.equal(normalise('  Related   Issue  '), 'related issue');
+});
+
+// ---------------------------------------------------------------------------
+// extractSection
+// ---------------------------------------------------------------------------
+
+test('extractSection returns content between a heading and the next one', () => {
+  const text = '## A\n\nfirst\n\n## B\n\nsecond\n\n## C\n\nthird';
+  assert.equal(extractSection(text, 'B').trim(), 'second');
+});
+
+test('extractSection returns everything to end of document for the last heading', () => {
+  const text = '## A\n\nfirst\n\n## B\n\nsecond';
+  assert.equal(extractSection(text, 'B').trim(), 'second');
+});
+
+test('extractSection is empty when the heading does not exist', () => {
+  const text = '## A\n\nfirst';
+  assert.equal(extractSection(text, 'Missing'), '');
+});
+
+test('extractSection ignores a heading-like line inside a fenced code block', () => {
+  const text = '## Format\n\n```\n## Checklist\nnot real\n```\n\n## Checklist\n\nreal content';
+  assert.equal(extractSection(text, 'Checklist').trim(), 'real content');
+});
+
+test('extractSection strips a fence closed with more backticks than it opened with', () => {
+  const text = '## Format\n\n```\n# not a real heading\n````\n\n## Checklist\n\nreal content';
+  assert.equal(extractSection(text, 'Checklist').trim(), 'real content');
+});
+
+// Regression: a shorter closing fence must NOT be treated as closing a
+// longer opener, or the real heading and content past the short "closer"
+// leak through as if the fence had already ended.
+test('a closer shorter than its opener does not end the fence early', () => {
+  const text = [
+    '## Checklist',
+    '',
+    '````',
+    '# not a real heading',
+    '```',
+    '## Also not a real heading',
+    '````',
+    '',
+    '- [x] Real item',
+  ].join('\n');
+  const section = extractSection(text, 'Checklist');
+  assert.doesNotMatch(section, /Also not a real heading/);
+  assert.match(section, /Real item/);
+});
+
+test('an unclosed fence runs to end of document, not just to end of section', () => {
+  const text = [
+    '## Checklist',
+    '',
+    '```',
+    '- [x] This item is inside an unclosed fence',
+  ].join('\n');
+  assert.equal(extractSection(text, 'Checklist').trim(), '');
+});
+
+// Regression: every regex here anchored `$` without the `m` flag, relying
+// on `.`/`.*` stopping at a lone `\n`. JavaScript's `.` does not match
+// `\r`, so a line ending in `\r` (a CRLF body, what the GitHub web editor
+// produces) matched nothing, and extractSection returned an empty section
+// for every heading. A fully conformant CRLF body was reported as though
+// every section had been deleted.
+test('checkConformance treats a CRLF body the same as the equivalent LF body', () => {
+  const lfBody = withSections({ aiDisclosure: 'AI assistance used: No' });
+  const crlfBody = lfBody.replace(/\n/g, '\r\n');
+  const result = checkConformance(REAL_TEMPLATE, crlfBody);
+  assert.equal(result.conformant, true);
+  assert.deepEqual(result.problems, []);
+});
+
+test('a CRLF body missing a checklist item is still caught, not silently passed', () => {
+  const items = extractChecklistItems(FILLED_CHECKLIST);
+  const gutted = items.slice(1).map((item) => `- [x] ${item}`).join('\n');
+  const lfBody = withSections({ checklist: gutted, aiDisclosure: 'AI assistance used: No' });
+  const result = checkConformance(REAL_TEMPLATE, lfBody.replace(/\n/g, '\r\n'));
+  assert.equal(result.conformant, false);
+  assert.match(result.problems[0], /^Checklist:/);
+});
+
+// ---------------------------------------------------------------------------
+// extractChecklistItems / missingChecklistItems
+// ---------------------------------------------------------------------------
+
+test('extractChecklistItems pulls the label text off each checkbox line', () => {
+  const text = '- [ ] First item\n- [x] Second item\nNot a checkbox line';
+  assert.deepEqual(extractChecklistItems(text), ['First item', 'Second item']);
+});
+
+test('extractChecklistItems also recognises numbered list markers', () => {
+  const text = '1. [ ] First item\n2. [x] Second item';
+  assert.deepEqual(extractChecklistItems(text), ['First item', 'Second item']);
+});
+
+test('an emoji-only template item is silently unenforceable, not a false pass', () => {
+  // Regression: the old heading matcher let an empty normalisation match
+  // everything, which is a bypass. Here the guard is the other shape: an
+  // item that normalises to nothing is dropped from the missing-list
+  // rather than ever being reportable, which is a safe degrade, not a
+  // bypass, since it can never cause a body to wrongly look conformant.
+  const missing = missingChecklistItems(['🚀', 'Do the real thing'], []);
+  assert.deepEqual(missing, ['Do the real thing']);
+});
+
+test('missingChecklistItems finds nothing when every template item has a body match', () => {
+  const missing = missingChecklistItems(['Do the thing'], ['Do the thing']);
+  assert.deepEqual(missing, []);
+});
+
+test('missingChecklistItems tolerates markdown link syntax around an item', () => {
+  const missing = missingChecklistItems(
+    ['I have read the [contribution guidelines](https://example.com/process)'],
+    ['I have read the [contribution guidelines](https://example.com/process)'],
+  );
+  assert.deepEqual(missing, []);
+});
+
+test('missingChecklistItems reports an item with no match at all', () => {
+  const missing = missingChecklistItems(['Do the thing', 'Do another thing'], ['Do the thing']);
+  assert.deepEqual(missing, ['Do another thing']);
+});
+
+test('the real template checklist has all 7 known items', () => {
+  const items = extractChecklistItems(extractSection(REAL_TEMPLATE, 'Checklist'));
+  assert.equal(items.length, 7);
+});
+
+// ---------------------------------------------------------------------------
+// checkAiDisclosure
+// ---------------------------------------------------------------------------
+
+test('the unedited template placeholder is not an answer', () => {
+  const problems = checkAiDisclosure(extractSection(REAL_TEMPLATE, 'AI Assistance disclosure'));
+  assert.equal(problems.length, 1);
+  assert.match(problems[0], /must be answered Yes or No/);
+});
+
+test('an empty AI Assistance disclosure section is not an answer', () => {
+  const problems = checkAiDisclosure('');
+  assert.equal(problems.length, 1);
+});
+
+test('"No" needs no further detail', () => {
+  const problems = checkAiDisclosure('AI assistance used: No\n');
+  assert.deepEqual(problems, []);
+});
+
+test('"Nope" is not recognised as an answer (word-boundary match only)', () => {
+  const problems = checkAiDisclosure('AI assistance used: Nope\n');
+  assert.equal(problems.length, 1);
+});
+
+// Regression: the placeholder guard used to be a single literal-string
+// check (`answer !== 'yes no'`), which only caught the exact unedited
+// text. Emphasising or striking one option, or reordering the two,
+// normalises to different text that still contains both a yes-token and a
+// no-token, and slipped through as an answer.
+test('emphasising one option instead of deleting it is still ambiguous', () => {
+  const problems = checkAiDisclosure('AI assistance used: **Yes** / No\n');
+  assert.equal(problems.length, 1);
+  assert.match(problems[0], /must be answered Yes or No/);
+});
+
+test('striking one option instead of deleting it is still ambiguous', () => {
+  const problems = checkAiDisclosure('AI assistance used: Yes / ~~No~~\n');
+  assert.equal(problems.length, 1);
+});
+
+test('reordering the placeholder is still ambiguous, not a considered "No"', () => {
+  const problems = checkAiDisclosure('AI assistance used: No / Yes\n');
+  assert.equal(problems.length, 1);
+  assert.match(problems[0], /must be answered Yes or No/);
+});
+
+test('"Yes / No (delete one)" is reported as ambiguous, not as 4 missing fields', () => {
+  const problems = checkAiDisclosure('AI assistance used: Yes / No (delete one)\n');
+  assert.equal(problems.length, 1);
+  assert.match(problems[0], /must be answered Yes or No/);
+});
+
+test('"None", "N/A" and "Not applicable" are recognised as No', () => {
+  for (const value of ['None', 'N/A', 'Not applicable']) {
+    const problems = checkAiDisclosure(`AI assistance used: ${value}\n`);
+    assert.deepEqual(problems, [], `expected "${value}" to need no further detail`);
+  }
+});
+
+test('"Yes" on its own is missing all four detail fields', () => {
+  const problems = checkAiDisclosure('AI assistance used: Yes\n');
+  assert.equal(problems.length, 4);
+  assert.match(problems[0], /Tool\(s\) used/);
+});
+
+test('"Yes" with every field answered on the same line has no problems', () => {
+  const body = [
+    'AI assistance used: Yes',
+    '',
+    'Tool(s) used: Claude Code',
+    '',
+    'Purpose of assistance: implementation',
+    '',
+    'Parts of the contribution affected: the whole diff',
+    '',
+    'Human validation performed: reviewed and tested',
+  ].join('\n');
+  assert.deepEqual(checkAiDisclosure(body), []);
+});
+
+test('"Yes" with fields answered on the following line also has no problems', () => {
+  const body = [
+    'AI assistance used: Yes',
+    '',
+    'Tool(s) used:',
+    'Claude Code',
+    '',
+    'Purpose of assistance:',
+    'implementation',
+    '',
+    'Parts of the contribution affected:',
+    'the whole diff',
+    '',
+    'Human validation performed:',
+    'reviewed and tested',
+  ].join('\n');
+  assert.deepEqual(checkAiDisclosure(body), []);
+});
+
+test('"Yes" with one field left blank reports only that field', () => {
+  const body = [
+    'AI assistance used: Yes',
+    '',
+    'Tool(s) used: Claude Code',
+    '',
+    'Purpose of assistance:',
+    '',
+    'Parts of the contribution affected: the whole diff',
+    '',
+    'Human validation performed: reviewed and tested',
+  ].join('\n');
+  const problems = checkAiDisclosure(body);
+  assert.equal(problems.length, 1);
+  assert.match(problems[0], /Purpose of assistance/);
+});
+
+test('a blank field followed by the next label, not an answer, is still blank', () => {
+  const body = [
+    'AI assistance used: Yes',
+    '',
+    'Tool(s) used:',
+    '',
+    'Purpose of assistance: implementation',
+    '',
+    'Parts of the contribution affected: the whole diff',
+    '',
+    'Human validation performed: reviewed and tested',
+  ].join('\n');
+  const problems = checkAiDisclosure(body);
+  assert.equal(problems.length, 1);
+  assert.match(problems[0], /Tool\(s\) used/);
+});
+
+// ---------------------------------------------------------------------------
+// checkConformance
+// ---------------------------------------------------------------------------
+
+test('a fully filled-in body is conformant', () => {
+  const body = withSections({
+    aiDisclosure: 'AI assistance used: No',
+  });
+  const result = checkConformance(REAL_TEMPLATE, body);
+  assert.equal(result.conformant, true);
+  assert.deepEqual(result.problems, []);
+});
+
+test('an unedited template body is not conformant (checklist ok, AI disclosure not answered)', () => {
+  const result = checkConformance(REAL_TEMPLATE, REAL_TEMPLATE);
+  assert.equal(result.conformant, false);
+  assert.equal(result.problems.length, 1);
+  assert.match(result.problems[0], /AI Assistance disclosure/);
+});
+
+test('a deleted checklist item and an unanswered disclosure are both reported', () => {
+  const items = extractChecklistItems(FILLED_CHECKLIST);
+  const gutted = items.slice(1).map((item) => `- [x] ${item}`).join('\n'); // drop the first item
+  const body = withSections({ checklist: gutted, aiDisclosure: 'AI assistance used: Yes / No' });
+  const result = checkConformance(REAL_TEMPLATE, body);
+  assert.equal(result.conformant, false);
+  assert.equal(result.problems.length, 2);
+  assert.match(result.problems[0], /^Checklist:/);
+  assert.match(result.problems[1], /^AI Assistance disclosure:/);
+});
+
+test('problem statement, related issue, proposed changes and format are never checked', () => {
+  const body = withSections({ aiDisclosure: 'AI assistance used: No' })
+    .replace(/## Problem Statement[\s\S]*?(?=\n## )/, '')
+    .replace(/## Related Issue[\s\S]*?(?=\n## )/, '')
+    .replace(/## Proposed Changes[\s\S]*?(?=\n## )/, '')
+    .replace(/## Format[\s\S]*?(?=\n## )/, '');
+  const result = checkConformance(REAL_TEMPLATE, body);
+  assert.equal(result.conformant, true);
+});
+
+test('checkConformance treats a null pull request body as missing everything, not a crash', () => {
+  const result = checkConformance(REAL_TEMPLATE, null);
+  assert.equal(result.conformant, false);
+  assert.ok(result.problems.length > 0);
+});
+
+// ---------------------------------------------------------------------------
+// closeMessage
+// ---------------------------------------------------------------------------
+
+test('closeMessage names every problem and the override label', () => {
+  const body = closeMessage(['Checklist: missing "Do the thing"']);
+  assert.match(body, /Do the thing/);
+  assert.match(body, new RegExp(OVERRIDE_LABEL));
+});
+
+// ---------------------------------------------------------------------------
+// run(): orchestration. readTemplate is injected so these never touch disk.
+// ---------------------------------------------------------------------------
+
+test('run() takes no action on a conformant pull request', async () => {
+  const github = fakeGithub();
+  const body = withSections({ aiDisclosure: 'AI assistance used: No' });
+  const result = await run({
+    core: fakeCore(),
+    github,
+    context: context({ pull_request: pr({ body }) }),
+    readTemplate: () => REAL_TEMPLATE,
+  });
+  assert.equal(result.action, 'none');
+  assert.equal(github.comments.length, 0);
+  assert.equal(github.closed.length, 0);
+});
+
+test('run() comments and closes a pull request with problems', async () => {
+  const github = fakeGithub();
+  const number = CUTOFF_PR_NUMBER + 42;
+  const result = await run({
+    core: fakeCore(),
+    github,
+    context: context({ pull_request: pr({ number, body: REAL_TEMPLATE }) }),
+    readTemplate: () => REAL_TEMPLATE,
+  });
+  assert.equal(result.action, 'closed');
+  assert.equal(result.problems.length, 1);
+  assert.equal(github.comments.length, 1);
+  assert.equal(github.comments[0].issue_number, number);
+  assert.match(github.comments[0].body, /AI Assistance disclosure/);
+  assert.equal(github.closed.length, 1);
+  assert.deepEqual(github.closed[0], {
+    owner: 'external-secrets', repo: 'external-secrets', pull_number: number, state: 'closed',
+  });
+});
+
+test('run() skips a pull request at or below the cutoff number, even with problems', async () => {
+  const github = fakeGithub();
+  const result = await run({
+    core: fakeCore(),
+    github,
+    context: context({ pull_request: pr({ number: CUTOFF_PR_NUMBER, body: '' }) }),
+    readTemplate: () => REAL_TEMPLATE,
+  });
+  assert.equal(result.action, 'skip-before-cutoff');
+  assert.equal(github.comments.length, 0);
+  assert.equal(github.closed.length, 0);
+});
+
+test('run() skips a bot author without writing anything', async () => {
+  const github = fakeGithub();
+  const result = await run({
+    core: fakeCore(),
+    github,
+    context: context({ pull_request: pr({ user: { login: 'dependabot[bot]' }, body: '' }) }),
+    readTemplate: () => REAL_TEMPLATE,
+  });
+  assert.equal(result.action, 'skip-bot');
+  assert.equal(github.comments.length, 0);
+  assert.equal(github.closed.length, 0);
+});
+
+test('run() skips a pull request carrying the override label', async () => {
+  const github = fakeGithub();
+  const result = await run({
+    core: fakeCore(),
+    github,
+    context: context({
+      pull_request: pr({ body: '', labels: [{ name: OVERRIDE_LABEL }] }),
+    }),
+    readTemplate: () => REAL_TEMPLATE,
+  });
+  assert.equal(result.action, 'skip-override');
+  assert.equal(github.comments.length, 0);
+  assert.equal(github.closed.length, 0);
+});
+
+test('run() fails loudly and takes no action when the template has no checklist items', async () => {
+  const github = fakeGithub();
+  const core = fakeCore();
+  const result = await run({
+    core,
+    github,
+    context: context({ pull_request: pr({ body: '' }) }),
+    readTemplate: () => 'no checklist in here at all',
+  });
+  assert.equal(result.action, 'error-empty-template');
+  assert.equal(core.failures.length, 1);
+  assert.equal(github.comments.length, 0);
+  assert.equal(github.closed.length, 0);
+});
+
+test('run() fails loudly and takes no action when the template cannot be read', async () => {
+  const github = fakeGithub();
+  const core = fakeCore();
+  const result = await run({
+    core,
+    github,
+    context: context({ pull_request: pr() }),
+    readTemplate: () => { throw new Error('ENOENT: no such file'); },
+  });
+  assert.equal(result.action, 'error-read-template');
+  assert.match(core.failures[0], /ENOENT/);
+  assert.equal(github.comments.length, 0);
+  assert.equal(github.closed.length, 0);
+});
+
+test('run() reads the real template from disk when readTemplate is not overridden', async () => {
+  // Exercises the default parameter, which is what actually runs in CI: every
+  // other test injects readTemplate and never touches this path.
+  const github = fakeGithub();
+  const body = withSections({ aiDisclosure: 'AI assistance used: No' });
+  const result = await run({
+    core: fakeCore(),
+    github,
+    context: context({ pull_request: pr({ body }) }),
+  });
+  assert.equal(result.action, 'none');
+});
+
+test('isBot matches known accounts and the generic [bot] suffix', () => {
+  assert.equal(isBot('dependabot'), true);
+  assert.equal(isBot('some-app[bot]'), true);
+  assert.equal(isBot('a-human-contributor'), false);
+  assert.equal(isBot(null), false);
+});
+
+// ---------------------------------------------------------------------------
+// Template drift: the AI disclosure field labels are hardcoded (see the
+// comment in pr-template-conformance.js on why), so if a maintainer renames
+// one in the template this test fails loudly instead of the check silently
+// checking for a label that no longer exists.
+// ---------------------------------------------------------------------------
+
+test('the real template still contains every hardcoded AI disclosure field label', () => {
+  const section = extractSection(REAL_TEMPLATE, 'AI Assistance disclosure');
+  for (const label of [AI_ASSISTANCE_LINE_LABEL, ...AI_DETAIL_FIELDS]) {
+    assert.match(
+      section,
+      new RegExp(`^[ \\t]*${label.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}[ \\t]*:`, 'im'),
+      `expected to find the label "${label}" in the real template`,
+    );
+  }
+});

+ 325 - 0
.github/scripts/pr-template-conformance.js

@@ -0,0 +1,325 @@
+/**
+ * Closes a pull request whose Checklist is missing an item, or whose AI
+ * Assistance disclosure hasn't actually been answered, so a contributor
+ * who guts the parts of the template maintainers rely on for trust is
+ * caught on open/reopen rather than discovered by a maintainer.
+ * See https://github.com/external-secrets/external-secrets/issues/6879.
+ *
+ * Deliberately narrow: only these two sections are checked. Problem
+ * Statement, Related Issue, Proposed Changes and Format are left alone,
+ * per the issue opener's own scoping in a maintainer discussion: "the rest
+ * will naturally sort itself out."
+ */
+
+import { readFileSync } from 'node:fs';
+
+// Accounts whose pull requests are never checked: their bodies are
+// generated by tooling, not copied from the contributor template.
+const BOTS = new Set(['dependabot', 'github-actions', 'eso-service-account-app']);
+
+export function isBot(login) {
+  return BOTS.has(login) || (typeof login === 'string' && login.endsWith('[bot]'));
+}
+
+// A maintainer applies this to push a pull request past the check: a
+// revert, a maintainer chore PR, or a false positive on the content match.
+export const OVERRIDE_LABEL = 'template-check-overridden';
+
+// Pull requests numbered at or below this were opened before the check
+// existed, so their authors never had a chance to write to it. This only
+// matters for `reopened`: an old pull request being reopened months later
+// must not be judged retroactively against a rule that postdates it. Set
+// to the highest pull request number in the repository on the day this
+// landed (external-secrets/external-secrets#6882).
+export const CUTOFF_PR_NUMBER = 6882;
+
+export const TEMPLATE_PATH = '.github/pull_request_template.md';
+
+// Identifies this workflow's comments; not read back yet, so a pull
+// request reopened without a fixed body gets one comment per attempt.
+const COMMENT_MARKER = '<!-- eso-pr-template-check -->';
+
+const CHECKLIST_HEADING = 'Checklist';
+const AI_DISCLOSURE_HEADING = 'AI Assistance disclosure';
+export const AI_ASSISTANCE_LINE_LABEL = 'AI assistance used';
+
+// The template's own free-text detail fields, asked only when assistance
+// was used. Hardcoded rather than parsed, since nothing in the template's
+// markup distinguishes a fillable field ("Tool(s) used:") from a plain
+// instructional line that also ends in a colon ("If yes provide details:").
+// Exported so the template-drift test in the test file checks these actual
+// constants, not a second hand-typed copy that could drift from them.
+export const AI_DETAIL_FIELDS = [
+  'Tool(s) used',
+  'Purpose of assistance',
+  'Parts of the contribution affected',
+  'Human validation performed',
+];
+
+/**
+ * CRLF collapses to LF before anything else runs. Every regex below anchors
+ * `$` to end-of-line without the `m` flag relying on `.`/`.*` stopping at a
+ * lone `\n`, and JavaScript's `.` does not match `\r`, so a line ending in
+ * `\r` (any CRLF body, which is what the GitHub web editor produces) failed
+ * every one of these patterns and made every section read as empty. That
+ * inverted the whole check: a fully conformant CRLF body got closed.
+ */
+function normaliseLineEndings(text) {
+  return text.replace(/\r\n?/g, '\n');
+}
+
+/**
+ * Drop fenced code blocks before section/heading scanning. Without this, a
+ * `#` comment inside a fence (the template's own Format section has one)
+ * would be read as a real heading. A single regex cannot express "the
+ * closer needs the same character and at least as many repeats as this
+ * particular opener": a backreference matches the opener's literal text,
+ * not a length to compare against, so independent `{3,}` quantifiers on
+ * each side let a shorter closer end a longer opener early, exposing
+ * whatever comes after it (including a real heading) as if it were normal
+ * text. Tracking fence state line by line, remembering the opener's actual
+ * character and length, closes that gap; an unclosed fence runs to end of
+ * document, matching how GitHub itself renders one.
+ */
+function stripFences(text) {
+  const kept = [];
+  let fence = null;
+  for (const line of text.split('\n')) {
+    if (fence) {
+      if (new RegExp(`^ {0,3}[${fence.char}]{${fence.len},}[ \\t]*$`).test(line)) fence = null;
+      continue;
+    }
+    const opened = line.match(/^ {0,3}(`{3,}|~{3,})/);
+    if (opened) {
+      fence = { char: opened[1][0], len: opened[1].length };
+      continue;
+    }
+    kept.push(line);
+  }
+  return kept.join('\n');
+}
+
+/**
+ * Lowercase, strip markdown emphasis and punctuation, collapse whitespace.
+ * Two headings or items that differ only in styling or trailing
+ * punctuation must still be treated as the same one.
+ */
+export function normalise(text) {
+  return text
+    .toLowerCase()
+    .replace(/[`*_]/g, '')
+    .replace(/[^\w\s]/g, '')
+    .replace(/\s+/g, ' ')
+    .trim();
+}
+
+/**
+ * Content of the first heading (any level) whose text normalises to
+ * `heading`, up to the next heading of any level or end of document. Empty
+ * if the heading isn't found at all, which is itself a finding: a section
+ * that was deleted outright has nothing in it to satisfy the checks below.
+ */
+export function extractSection(text, heading) {
+  const lines = stripFences(text).split('\n');
+  const target = normalise(heading);
+  let capturing = false;
+  const collected = [];
+  for (const line of lines) {
+    const m = line.match(/^#{1,6}[ \t]+(.+)$/);
+    if (m) {
+      if (capturing) break;
+      if (normalise(m[1]) === target) capturing = true;
+      continue;
+    }
+    if (capturing) collected.push(line);
+  }
+  return collected.join('\n');
+}
+
+/**
+ * Verbatim label text of each checklist item in `text`, at any indent
+ * depth, whether bulleted or numbered (GitHub renders both as task items).
+ */
+export function extractChecklistItems(text) {
+  const marker = '(?:[-*+]|\\d+\\.)';
+  return (text.match(new RegExp(`^[ \\t]*${marker}[ \\t]+\\[[ xX]\\][ \\t]+.+$`, 'gm')) || [])
+    .map((line) => line.replace(new RegExp(`^[ \\t]*${marker}[ \\t]+\\[[ xX]\\][ \\t]+`), '').trim());
+}
+
+/**
+ * Template checklist items with no match anywhere in the pull request
+ * body's own checklist. Checklist item text is long and specific enough
+ * (a full sentence) that a plain substring check carries none of the
+ * short-heading collision risk a single word like "Format" would.
+ */
+export function missingChecklistItems(templateItems, bodyItems) {
+  const body = bodyItems.map(normalise).filter((b) => b.length > 0);
+  return templateItems.filter((expected) => {
+    const e = normalise(expected);
+    return e.length > 0 && !body.some((b) => b.includes(e));
+  });
+}
+
+/**
+ * The value on a `label: value` line in `text`, or on the next non-blank
+ * line if the label line itself has nothing after the colon (the template
+ * presents each field this way: label alone, blank line, room to answer).
+ * Null if the label doesn't appear at all; '' if it appears but is empty.
+ */
+function fieldValue(text, label) {
+  const escaped = label.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+  const labelLine = new RegExp(`^[ \\t]*${escaped}[ \\t]*:[ \\t]*(.*)$`, 'i');
+  const knownLabels = [AI_ASSISTANCE_LINE_LABEL, ...AI_DETAIL_FIELDS].map(normalise);
+  const lines = text.split('\n');
+  for (let i = 0; i < lines.length; i += 1) {
+    const m = lines[i].match(labelLine);
+    if (!m) continue;
+    if (m[1].trim()) return m[1].trim();
+    for (let j = i + 1; j < lines.length; j += 1) {
+      const next = lines[j].trim();
+      if (next === '') continue;
+      if (/^#{1,6}[ \t]/.test(next)) return '';
+      // The next non-blank line starts a different field (whether or not
+      // that field's own answer sits on the same line), not a continuation
+      // of this one, whenever the text before its first colon is itself a
+      // known label.
+      const colonIndex = next.indexOf(':');
+      const beforeColon = colonIndex === -1 ? next : next.slice(0, colonIndex);
+      if (knownLabels.includes(normalise(beforeColon))) return '';
+      return next;
+    }
+    return '';
+  }
+  return null;
+}
+
+// Answers that mean "No" without containing the word "no" as a standalone
+// token (so "None"/"N/A" aren't mistaken for the unedited/ambiguous case
+// the way "No / Yes" would be, and aren't rejected as gibberish the way
+// "Nope" correctly is).
+const NO_LIKE_PREFIX = /^(none|na|n a|not applicable)\b/;
+
+/**
+ * Checks the AI Assistance disclosure section. Requires the top-level
+ * question to be answered Yes or No, not left ambiguous, and if Yes,
+ * requires each detail field to carry real content rather than being left
+ * blank.
+ *
+ * "Ambiguous" is a shape, not one literal string: the template's own
+ * placeholder ("Yes / No") is the common case, but a contributor who
+ * emphasises or strikes through one option instead of deleting it, or who
+ * simply reorders it ("No / Yes"), normalises to text that still contains
+ * both a yes-token and a no-token. Any such text is treated the same as
+ * the untouched placeholder, since a maintainer glancing at it could not
+ * tell which one was chosen either. The known false-positive this trades
+ * for: "Yes, no concerns" would also read as ambiguous.
+ */
+export function checkAiDisclosure(sectionText) {
+  const raw = fieldValue(sectionText, AI_ASSISTANCE_LINE_LABEL);
+  const answer = raw ? normalise(raw) : '';
+  const hasYes = /\byes\b/.test(answer);
+  const hasNo = /\bno\b/.test(answer) || NO_LIKE_PREFIX.test(answer);
+  const ambiguous = hasYes && hasNo;
+  const isYes = hasYes && !ambiguous;
+  const isNo = hasNo && !ambiguous;
+
+  if (!isYes && !isNo) {
+    return [`"${AI_ASSISTANCE_LINE_LABEL}" must be answered Yes or No`];
+  }
+  if (!isYes) return [];
+
+  return AI_DETAIL_FIELDS
+    .filter((field) => !fieldValue(sectionText, field))
+    .map((field) => `"${field}" must be filled in since AI assistance was Yes`);
+}
+
+/**
+ * Decides conformance and, on failure, what's wrong. A pure function so
+ * the decision itself can be tested without a github client.
+ */
+export function checkConformance(templateBody, prBody) {
+  const template = normaliseLineEndings(templateBody);
+  const body = normaliseLineEndings(prBody || '');
+  const templateChecklist = extractChecklistItems(extractSection(template, CHECKLIST_HEADING));
+  const bodyChecklist = extractChecklistItems(extractSection(body, CHECKLIST_HEADING));
+  const missingChecklist = missingChecklistItems(templateChecklist, bodyChecklist);
+  const aiProblems = checkAiDisclosure(extractSection(body, AI_DISCLOSURE_HEADING));
+
+  const problems = [
+    ...missingChecklist.map((item) => `Checklist: missing "${item}"`),
+    ...aiProblems.map((p) => `AI Assistance disclosure: ${p}`),
+  ];
+
+  return {
+    conformant: problems.length === 0,
+    problems,
+    templateChecklistCount: templateChecklist.length,
+  };
+}
+
+export function closeMessage(problems) {
+  return [
+    COMMENT_MARKER,
+    `This pull request does not fully match the required template at \`${TEMPLATE_PATH}\`:`,
+    '',
+    ...problems.map((p) => `- ${p}`),
+    '',
+    'Closing so the description can be completed from the template. Fix the body and reopen this pull request once every item above is addressed, or open a new one.',
+    '',
+    `If this is a false positive, ask a maintainer for the \`${OVERRIDE_LABEL}\` label, then reopen.`,
+  ].join('\n');
+}
+
+export default async function run({
+  core, github, context, readTemplate = () => readFileSync(TEMPLATE_PATH, 'utf8'),
+}) {
+  const { owner, repo } = context.repo;
+  const pr = context.payload.pull_request;
+
+  if (pr.number <= CUTOFF_PR_NUMBER) {
+    core.info(`PR #${pr.number}: at or below the cutoff (${CUTOFF_PR_NUMBER}), skipping`);
+    return { action: 'skip-before-cutoff' };
+  }
+
+  const author = pr.user ? pr.user.login : null;
+  if (isBot(author)) {
+    core.info(`PR #${pr.number}: author ${author} is a bot, skipping`);
+    return { action: 'skip-bot' };
+  }
+
+  const labels = (pr.labels || []).map((l) => l.name);
+  if (labels.includes(OVERRIDE_LABEL)) {
+    core.info(`PR #${pr.number}: ${OVERRIDE_LABEL} present, skipping`);
+    return { action: 'skip-override' };
+  }
+
+  let templateBody;
+  try {
+    templateBody = readTemplate();
+  } catch (error) {
+    core.setFailed(`Could not read the PR template at ${TEMPLATE_PATH}: ${error.message}`);
+    return { action: 'error-read-template' };
+  }
+
+  const { conformant, problems, templateChecklistCount } = checkConformance(templateBody, pr.body);
+  if (templateChecklistCount === 0) {
+    core.setFailed(
+      `No checklist items found in ${TEMPLATE_PATH}; refusing to check pull requests `
+      + 'against a template that failed to parse.',
+    );
+    return { action: 'error-empty-template' };
+  }
+  if (conformant) {
+    core.info(`PR #${pr.number}: conforms to the template`);
+    return { action: 'none' };
+  }
+
+  core.info(`PR #${pr.number}: ${problems.length} problem(s), closing`);
+  await github.rest.issues.createComment({
+    owner, repo, issue_number: pr.number, body: closeMessage(problems),
+  });
+  await github.rest.pulls.update({
+    owner, repo, pull_number: pr.number, state: 'closed',
+  });
+  return { action: 'closed', problems };
+}

+ 45 - 0
.github/workflows/pr-template-conformance.yml

@@ -0,0 +1,45 @@
+# Closes a pull request that has dropped a section heading from
+# .github/pull_request_template.md, so a contributor who skipped the
+# template is caught on open/reopen rather than discovered by a maintainer.
+# See https://github.com/external-secrets/external-secrets/issues/6879.
+#
+# zizmor flags pull_request_target as a dangerous trigger on sight. It is
+# used here in its documented safe form (see review-state.yml for the same
+# pattern): the checkout takes the base ref with no `ref:` and a sparse
+# path, so no pull request code is ever fetched or executed. The check
+# itself reads the pull request body from the event payload, not from the
+# checked-out tree.
+name: PR Template Conformance
+
+on: # zizmor: ignore[dangerous-triggers] base-ref checkout only, no PR code is executed
+  pull_request_target:
+    types: [opened, reopened]
+
+permissions:
+  contents: read
+
+jobs:
+  check:
+    name: Check PR body against the template
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write
+      issues: write
+    steps:
+      - name: Checkout scripts and template from the base ref
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          sparse-checkout: |
+            .github/scripts
+            .github/pull_request_template.md
+          persist-credentials: false
+
+      - name: Check template conformance
+        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+        with:
+          script: |
+            const { default: run } = await import(
+              `${process.env.GITHUB_WORKSPACE}/.github/scripts/pr-template-conformance.js`
+            );
+            await run({ core, github, context });

+ 5 - 0
docs/contributing/process.md

@@ -58,6 +58,11 @@ for the lifecycle of the PR: review, merging, ping on inactivity, close.
 We close pull requests or issues if there is no response from the author for
 a period of time. Feel free to reopen if you want to get back on it.
 
+Pull requests are also closed automatically on open or reopen if their
+description has dropped a section from `.github/pull_request_template.md`.
+Fix the description from the template and reopen, or open a new pull request,
+once every section is back.
+
 _Note:_
 Pull requests that are labelled with _size/l_ and above _MUST_ have at least **TWO**
 approvers for it to be merged. Please respect this policy to ensure the quality