oh-my-opencode-slim/src/tools/smartfetch/codemap.md

src/tools/smartfetch/

Responsibility

• Implement the built-in webfetch tool: fetch remote documents, enforce redirect/origin policy, probe llms.txt when useful, and return normalized text/markdown/html output (tool.ts, network.ts).
• Handle content shaping around that fetch step: HTML extraction, metadata/frontmatter rendering, heading cleanup, cache keying, binary persistence, and secondary-model fallback (utils.ts, cache.ts, binary.ts, secondary-model.ts).

Design Patterns and Decisions

• One orchestration entrypoint: createWebfetchTool in tool.ts owns permission prompts, cache lookup/revalidation, llms.txt preference logic, binary-vs-text branching, metadata emission, and optional secondary-model summarization.
• Transport/policy split from rendering: network.ts focuses on URL normalization, redirect allowlists, charset/body decoding, header extraction, and llms.txt probing, while utils.ts focuses on turning fetched content into cleaned text/markdown/html plus frontmatter and user-facing messages.
• Cache keyed by fetch shape: cache.ts keys fetches by URL plus behavior-affecting options (extract_main, prefer_llms_txt, save_binary), while render format is derived from the cached fetch result so text/markdown/html do not force redundant network requests.
• Graceful degradation: missing/invalid llms.txt, blocked redirects, metadata-only binary responses, and secondary-model failures all return a usable result instead of throwing away the fetched content.

Data & Control Flow

1. createWebfetchTool normalizes the requested URL, derives permission patterns/allowed origins, asks for webfetch permission, and computes the cache key (tool.ts, network.ts, cache.ts).

2. If prefer_llms_txt applies, probeLlmsText tries /llms-full.txt then /llms.txt, following only permitted redirects and rejecting HTML/login-wall responses (network.ts).

3. When the tool falls back to the page itself, fetchWithUpgradeFallback handles HTTPS upgrade fallback, redirect enforcement, conditional headers for revalidation, binary detection, and bounded body reads (network.ts, tool.ts).

4. Text/HTML payloads are decoded and normalized through extractFromHtml, cleanFetchedMarkdown, extractHeadingsFromMarkdown, frontmatter, and joinRenderedContent; binary payloads optionally persist via saveBinary and return a metadata message (utils.ts, binary.ts, tool.ts).

5. If the caller supplied a prompt and configured secondary models, runSecondaryModelWithFallback truncates input to a bounded size, disables tool access for the helper session, retries across configured models, and the tool degrades back to base fetched content if that step fails (secondary-model.ts, tool.ts).

Integration Points

• src/index.ts registers the tool under the public name webfetch, so agents can call it alongside council and AST-grep tools.
• src/tools/smartfetch/index.ts re-exports the tool factory, description, and shared types for other modules or docs to import without reaching into implementation files.
• secondary-model.ts depends on the OpenCode plugin client (PluginInput['client']) to spawn an isolated helper session, resolve small_model from the effective OpenCode config, and resolve explorer / librarian fallbacks from slim's own plugin config loader.
• cache.ts, network.ts, and utils.ts are intentionally reusable seams for tests: cache behavior, redirect policy, llms probing, heading extraction, and render/metadata helpers can be verified without hitting the full tool entrypoint.