Browse Source

fix(council): harden synthesis agent to deny-all and sync stale tests

Council agent is text-in/text-out only; replace read-only permission
with createSynthesisOnlyPermission (deny-all tools). Update the stale
index test to assert deny-all, and refresh the cache-impact snapshot
for the council description text changed earlier on this branch.
Michael Henke 4 weeks ago
parent
commit
605f48c3e3

+ 3 - 6
src/agents/council.ts

@@ -1,6 +1,5 @@
-import { READONLY_FILE_OPERATIONS_RULES } from '../config';
 import { type AgentDefinition, resolvePrompt } from './orchestrator';
-import { createReadOnlyAgentPermission } from './permissions';
+import { createSynthesisOnlyPermission } from './permissions';
 
 // NOTE: Councillor system prompts live in the councillor agent factory.
 // The council agent synthesizes councillor responses passed by the orchestrator.
@@ -12,7 +11,7 @@ synthesizer for multi-model consensus.
 
 **Role**: You receive raw responses from multiple councillors (different models) and synthesize them into a structured council report. You do NOT dispatch councillors yourself - the orchestrator handles dispatch and provides the councillor results.
 
-**Tools**: You have read-only codebase inspection tools. You do not have write, edit, shell, or task tools.
+**Tools**: You have NO tools. You synthesize purely from the councillor responses provided in your context. Do not read, glob, grep, or run shell commands.
 
 **Synthesis Process** (MANDATORY - follow in order):
 1. Read the original user prompt (provided in the context)
@@ -29,8 +28,6 @@ key insight and unique contribution by name
 - Do not omit per-councillor details from the final response
 - Don't just average responses - choose the best approach and improve upon it
 
-${READONLY_FILE_OPERATIONS_RULES}
-
 **Required Output Format**:
 Always include these sections in your final response:
 
@@ -76,7 +73,7 @@ export function createCouncilAgent(
       temperature: 0.1,
       prompt,
       permission: {
-        ...createReadOnlyAgentPermission(),
+        ...createSynthesisOnlyPermission(),
       },
     },
   };

+ 9 - 5
src/agents/index.test.ts

@@ -362,23 +362,27 @@ describe('tool permissions', () => {
     ).toBe('deny');
   });
 
-  test('council agent has synthesis-only read-only permissions', () => {
+  test('council agent has synthesis-only (deny-all) permissions', () => {
     const agents = createAgents({
       council: councilConfig(),
     });
     const council = agents.find((a) => a.name === 'council');
     const permission = council?.config.permission as Record<string, string>;
     expect(permission['*']).toBe('deny');
-    expect(permission.read).toBe('allow');
-    expect(permission.glob).toBe('allow');
-    expect(permission.grep).toBe('allow');
-    expect(permission.ast_grep_search).toBe('allow');
+    expect(permission.read).toBe('deny');
+    expect(permission.glob).toBe('deny');
+    expect(permission.grep).toBe('deny');
+    expect(permission.ast_grep_search).toBe('deny');
+    expect(permission.codesearch).toBe('deny');
+    expect(permission.lsp).toBe('deny');
+    expect(permission.list).toBe('deny');
     expect(permission.bash).toBe('deny');
     expect(permission.edit).toBe('deny');
     expect(permission.write).toBe('deny');
     expect(permission.apply_patch).toBe('deny');
     expect(permission.ast_grep_replace).toBe('deny');
     expect(permission.task).toBe('deny');
+    expect(permission.question).toBe('deny');
   });
 
   test('councillor remains read-only after default permissions are applied', () => {

+ 27 - 0
src/agents/permissions.ts

@@ -29,3 +29,30 @@ export function createReadOnlyAgentPermission(): AgentPermission {
     ast_grep_search: 'allow',
   } as AgentPermission;
 }
+
+/**
+ * Strict deny-all permissions for the council synthesis agent.
+ *
+ * The council agent is text-in/text-out only — it must NOT use any
+ * file-inspection tools. Councillors already perform codebase exploration;
+ * the council only reconciles their text output.
+ */
+export function createSynthesisOnlyPermission(): AgentPermission {
+  return {
+    '*': 'deny',
+    bash: 'deny',
+    edit: 'deny',
+    write: 'deny',
+    apply_patch: 'deny',
+    ast_grep_replace: 'deny',
+    task: 'deny',
+    question: 'deny',
+    read: 'deny',
+    glob: 'deny',
+    grep: 'deny',
+    lsp: 'deny',
+    list: 'deny',
+    codesearch: 'deny',
+    ast_grep_search: 'deny',
+  } as AgentPermission;
+}

+ 3 - 3
src/hooks/__snapshots__/cache-payload.snapshot.test.ts.snap

@@ -72,14 +72,14 @@ You have perfect understanding of agent's context management, understand well th
 
 @council
 - Lane: High-stakes multi-model decision support
-- Role: Multi-LLM consensus engine that runs several councillors, synthesizes their views, and returns a structured council report.
+- Role: Multi-LLM consensus engine that receives raw councillor responses and synthesizes them into a structured council report.
 - Permissions: Read files
 - Stats: 3x slower than orchestrator, 3x or more cost of orchestrator
-- Capabilities: Runs multiple models in parallel, compares their answers, resolves disagreements, and produces a final synthesized answer plus councillor details and consensus summary.
+- Capabilities: Synthesizes responses from independently-dispatched councillors, compares their answers, resolves disagreements, and produces a final synthesized answer plus councillor details and consensus summary.
 - **Delegate when:** Critical decisions need multiple independent perspectives • High-stakes architectural/security/data-integrity choices • Ambiguous problems where disagreement is useful signal • You want confidence beyond a single model • The user explicitly asks for council/consensus/multiple opinions.
 - **Don't delegate when:** Straightforward tasks you're confident about • Speed matters more than confidence • Routine implementation/debugging • A single specialist is clearly the right tool • You only need current docs/search/code review rather than multi-model consensus.
 - **How to call:** Send the full question/task and relevant context. Be explicit about what decision, trade-off, or answer the council should resolve. Do not ask council to do routine code edits.
-- **Result handling:** Council returns a structured response that may include: synthesized Council Response, individual Councillor Details, and Council Summary/confidence. Preserve that structure when the user asked for council output. Do not pretend the council only returned a final answer. If you need to act on the council result, first briefly state the council's recommendation, then proceed.
+- **Result handling:** Council returns a structured response that may include: synthesized Council Response, individual Per-Councillor Details, and Council Summary/confidence. Preserve that structure when the user asked for council output. Do not pretend the council only returned a final answer. If you need to act on the council result, first briefly state the council's recommendation, then proceed.
 - **Rule of thumb:** Need second/third opinions from different models? → @council. Need one expert lane? → use the specialist. Need final synthesis? → handle directly.
 
 @observer