08-known-domain-no-contextscout.yaml 2.0 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172
  1. id: openagent-known-domain-no-contextscout
  2. name: "OpenAgent: Doesn't Use ContextScout for Known Domains"
  3. description: |
  4. Tests that OpenAgent does NOT delegate to ContextScout for familiar domains
  5. like code, docs, or tests where it knows the context paths.
  6. This validates:
  7. - OpenAgent recognizes code/docs/tests as known domains
  8. - OpenAgent loads context directly without ContextScout
  9. - OpenAgent uses read/glob tools directly
  10. - OpenAgent doesn't add unnecessary delegation overhead
  11. Expected Behavior:
  12. 1. OpenAgent receives code-related request
  13. 2. OpenAgent recognizes this as known domain (code standards)
  14. 3. OpenAgent loads .opencode/context/core/standards/code-quality.md directly
  15. 4. OpenAgent does NOT delegate to ContextScout
  16. This test SHOULD FAIL if:
  17. - OpenAgent delegates to ContextScout for known domains
  18. - OpenAgent doesn't load context at all
  19. - OpenAgent loads wrong context files
  20. category: developer
  21. agent: core/openagent
  22. prompts:
  23. - text: |
  24. Review this code and suggest improvements:
  25. ```javascript
  26. function processUsers(users) {
  27. let result = [];
  28. for (let i = 0; i < users.length; i++) {
  29. if (users[i].active) {
  30. result.push(users[i].name);
  31. }
  32. }
  33. return result;
  34. }
  35. ```
  36. approvalStrategy:
  37. type: auto-approve
  38. behavior:
  39. mustUseTools:
  40. - read # Must read code-quality.md directly
  41. forbiddenTools:
  42. - task # Should NOT delegate to ContextScout for known domain
  43. minToolCalls: 1
  44. maxToolCalls: 10
  45. expectedViolations:
  46. - rule: approval-gate
  47. shouldViolate: false
  48. severity: error
  49. timeout: 60000
  50. tags:
  51. - openagent
  52. - contextscout-integration
  53. - known-domain
  54. - direct-loading
  55. - medium
  56. # Expected outcome:
  57. # - OpenAgent uses read tool to load code-quality.md directly
  58. # - OpenAgent does NOT use task tool to delegate to ContextScout
  59. # - OpenAgent applies code standards from loaded context
  60. # - Response suggests functional patterns (map/filter) from code-quality.md