Agent Validator Plugin

Validates that OpenAgent follows its defined prompt rules and execution patterns.

Features

✅ Tracks tool usage in real-time
✅ Validates approval gate enforcement
✅ Checks lazy context loading
✅ Analyzes delegation decisions (4+ file rule)
✅ Detects critical rule violations (auto-fix attempts)

Available Tools

`validate_session`

Validate the current agent session against defined rules.

validate_session

Options:

include_details (boolean, optional) - Include detailed evidence for each check

Returns: Validation report with compliance score

`check_approval_gates`

Check if approval gates were properly enforced before execution operations.

check_approval_gates

Returns: Approval gate compliance status

`export_validation_report`

Export a comprehensive validation report to a markdown file.

export_validation_report

Options:

output_path (string, optional) - Path to save the report (defaults to .tmp/validation-{sessionID}.md)

Returns: Path to exported report + summary

`analyze_delegation`

Analyze whether delegation decisions followed the 4+ file rule.

analyze_delegation

Returns: Delegation analysis with file count statistics

Validation Rules

The plugin checks for:

approval_gate_enforcement - Did agent request approval before bash/write/edit/task?
stop_on_failure - Did agent stop on errors or try to auto-fix?
lazy_context_loading - Did agent only load context files when needed?
delegation_appropriateness - Did agent delegate when 4+ files involved?
tool_usage - Track all tool calls for analysis

Usage Examples

Basic Validation

You: "Create a new API endpoint"
[Agent works on task]
You: "validate_session"

Check Approval Compliance

You: "Run the tests"
Agent: "Approval needed before proceeding."
You: "Approved. Also check_approval_gates"

Export Report

You: "We just finished refactoring. Export validation report"
Agent: [Exports to .tmp/validation-{id}.md]

Installation

The plugin auto-loads from .opencode/plugins/ when OpenCode starts.

Install dependencies:

cd .opencode/plugins
npm install
# or
bun install

How It Works

Event Tracking - Hooks into OpenCode SDK events:
- session.message.created
- tool.execute.before
- tool.execute.after
Behavior Analysis - Analyzes messages for:
- Tool invocations
- Approval language
- Context file reads
- Delegation patterns
Validation - Compares actual behavior against OpenAgent rules
Reporting - Generates compliance reports with scores and evidence

Compliance Scoring

100% - Perfect compliance
90-99% - Excellent (minor warnings)
80-89% - Good (some warnings)
70-79% - Fair (multiple warnings)
<70% - Needs improvement (errors or many warnings)

Troubleshooting

"No execution operations tracked"

Plugin just loaded, no prior tracking
Run a task first, then validate

"Error fetching session"

Check OpenCode SDK connection
Verify session ID is valid

False positives on approval gates

Agent may use different approval phrasing
Check approvalKeywords in plugin code
Add custom patterns if needed

Customization

Edit agent-validator.ts to:

Add custom validation rules
Modify approval detection patterns
Adjust delegation thresholds
Change severity levels

Next Steps

Test with simple sessions
Identify false positives/negatives
Refine validation logic
Add project-specific rules
Integrate into OpenAgent workflow

README.md 3.6 KB History Raw