MCP tools contract#

The MCP server (packages/mcp/src/server.ts) is how agents use vid2context. Claude Code, Claude Desktop, Cursor and VS Code connect over stdio (vid2context-mcp). ChatGPT and Claude.ai connect to the remote server over Streamable HTTP (/mcp, served by packages/mcp-http, item 45). Both transports serve exactly the same tools from createServer(registry, ctx).

Transports#

Item 45 owns this section.

Streamable HTTP rules:

• Every request builds its own Ctx with makeCtx(req). In stateless mode an aborted request closes its server, which aborts running tool calls. A session's server keeps the Ctx of the request that opened it, minus that request's signal; tool calls use the SDK's per-call signal.
• One library Storage is opened once and passed to every createServer(registry, ctx, version, { storage, ...serverOptions }).
• 401 responses carry WWW-Authenticate: Bearer resource_metadata="<origin>/.well-known/oauth-protected-resource/mcp", scope="mcp" when OAuth is on. 401, 403 and 429 bodies use the HTTP API error shape and are returned by the sub-app, never thrown.
• An Origin header not listed in VID2CONTEXT_CORS_ORIGINS (or the public origin) gets 403. When bound to loopback, a Host header that is not loopback, the public host or VID2CONTEXT_MCP_ALLOWED_HOSTS gets 403.
• A session is bound to the credential that opened it. Another credential gets 404, as if there were no session.
• Per-token token bucket: VID2CONTEXT_MCP_RATE_LIMIT requests per minute (default 120), answering 429 with Retry-After.
• Tools that touch the server's own filesystem (export_obsidian) refuse calls that arrive over HTTP.
• Discovery is public: /mcp/server-card (SEP-2127) and /.well-known/ai-catalog.json. OAuth endpoints are /.well-known/oauth-protected-resource[/mcp], /.well-known/oauth-authorization-server and /oauth/{register,authorize,token,revoke}. See docs/guides/remote-mcp.md.
• Protocol versions: the SDK in use (1.32.2) speaks 2025-11-25 and older. The 2026-07-28 changes (no sessions, no GET stream, Mcp-Method headers) follow when the SDK supports them.

Rules for every tool#

• Names are snake_case verb phrases. They are final once listed here; integrations, docs and the web app refer to them.

• Inputs: a video input is always called url (a URL, a YouTube id or a local path). Several inputs are urls: string[]. A collection is collection (its id). Formats come from the core formatters map at runtime, never from a hard-coded list.

• Outputs are text content. Text-format transcripts start with the header from contextHeader() (title, channel, URL, duration, transcript origin, chapters) so the agent always knows the source. Structured results are pretty-printed JSON text.

• Citations: wherever a result points into a video, include [m:ss] timestamps and, where the platform supports it, a timestamp URL from timestampUrl.

• Errors are tool results with isError: true and this JSON text, never protocol errors:

  jsonCopy

  { "error": "not-found", "message": "...", "hint": "...", "attempts": [] }

• Context budget: speech is about 12k tokens per hour. Tools that can return whole transcripts must also offer a way to fetch less (get_outline, get_section, search_transcript).

• Per-call context: each call runs with its own Ctx carrying the call's abort signal. Never keep per-request state on the server object.

• Synthesis is the agent's job by default. Tools return data and the server offers prompts that tell the agent how to build a cited brief. Server-side LLM synthesis is opt-in (item 32) for collections too large for the agent's context.

Implemented tools#

Every tool has a JSON Schema input with a description per field, annotations (readOnlyHint, openWorldHint, idempotentHint, plus destructiveHint: false on writes), and a description that says when to use it and roughly what it costs in tokens. Tools marked "structured" also declare an outputSchema and return the same object as structuredContent and as JSON text. Every video read through any tool is saved to the server's storage (the enabled storage provider, else MemoryStorage), so search_transcript, the collection tools and the resources see it.

createServer(registry, ctx, version?, opts?) holds no transport code. opts.storage shares one Storage with every tool and wired package (bin.ts opens the registry's backend up front); opts.extractAudio is the audio extractor for watch_live (bin.ts passes the ffmpeg one from @vid2context/live/node).

Defined in packages/mcp (item 44)#

Long operations over many videos (export_context_pack with list urls) send notifications/progress per video when the client sends a progress token. One failing video never fails the batch: it is reported in failed.

Wired from other packages#

packages/mcp/src/wire.ts calls each package's registration function with the same registry, base Ctx and storage.

The rows for these tools in the tables below describe their inputs and outputs. ask returns [n] Title @ m:ss <timestamp URL> source lines after the answer, and its error body nests the code as { error: { code, message, hint }, attempts }.

Adding a tool from another package#

The Owner column names the work item (see docs/ROADMAP.md) that implements the tool.

Ownership rule for parallel work: only item 44 edits packages/mcp. An item whose own package backs a tool exports a registration function from src/mcp.ts in that package:

// packages/ask/src/mcp.ts
import type { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import type { Ctx, Registry } from '@vid2context/core';
export function registerAskTools(server: McpServer, deps: { registry: Registry; ctx: Ctx }): void { /* server.registerTool('ask', ...) */ }

A registration function that lands later is wired with a one-line additive change to packages/mcp/src/wire.ts. Item 45 (packages/mcp-http) serves the same createServer() over Streamable HTTP and never defines tools of its own.

Every tool in the original plan is now implemented or wired. These are the inputs the wired tools take:

Collection tools (item 41)#

Refresh, watch lists, webhooks and import/export are on the HTTP API and the library package; see guides/collections.md.

Prompts (implemented)#

Both tell the calling agent to plan reading from token counts, read outline first (get_outline, get_section, search_transcript) or whole when it fits, and cite every claim as [Video title @ m:ss](timestamp link), flagging consensus, disagreement and single-source claims. Prompt arguments are strings, so urls is a comma, space or newline separated list. The text lives in packages/mcp/src/prompts.ts until @vid2context/insights (item 32) exports prompt text.

Resources (implemented)#

Listed from storage (newest 500 saved videos; every collection). Reading an id that is not saved is a not-found error with a hint.

Wave 2 tools (prompts 51 to 62)#

Same rules as above. Each package registers its tools from its own src/mcp.ts (register<Name>Tools(server, deps)); item 44, or the integration pass, wires them into createServer (all rows below are wired, see "Wired from other packages"). The package owner is the prompt number.