| 1234567891011121314151617181920212223242526272829303132333435363738394041424344 |
- id: ctx-simple-documentation-format
- name: "Context Loading: Documentation Format"
- description: |
- Simple test: Ask about documentation format and verify agent loads context file before responding.
-
- Expected behavior:
- - Agent should read docs.md or documentation.md BEFORE responding
- - Agent should reference project-specific documentation standards
-
- Files created: None (read-only test)
- Working directory: evals/test_tmp/
- category: developer
- agent: openagent
- model: anthropic/claude-sonnet-4-5
- # Single prompt test
- prompt: "What format should I use for documentation in this project?"
- # Expected behavior
- behavior:
- mustUseAnyOf:
- - [read] # Must use read tool
- requiresContext: true # Must load context before responding
- minToolCalls: 1 # At least one read
- # Expected violations
- expectedViolations:
- - rule: context-loading
- shouldViolate: false
- severity: error
- description: Must load docs.md or documentation.md before responding
- # Approval strategy
- approvalStrategy:
- type: auto-approve
- timeout: 60000
- tags:
- - context-loading
- - simple-test
- - read-only
- - documentation
|
