Agent clients

Use vid2context in Claude Desktop#

Claude Desktop can read any video through the vid2context MCP server: the transcript with timestamps, the metadata and the chapters. No key is needed. Captions come straight from the platform, and when a video has none, speech-to-text falls back through free and keyless Whisper rungs.

Install#

These steps use a clone of the repo (the npm package is not published yet, so these steps run the server from source).

  1. Open Settings, Developer, Edit Config. The file is ~/Library/Application Support/Claude/claude_desktop_config.json on macOS and %APPDATA%\Claude\claude_desktop_config.json on Windows.

  2. Clone the repo and install it once (Node.js 22 or newer). Run the next commands from the repo root.

    sh
    git clone https://github.com/nirholas/vid2context.git
    cd vid2context
    npm install
  3. Print the config with this clone's paths, then add it to claude_desktop_config.json. Merge it into mcpServers if the file already has other servers.

    sh
    printf '{"mcpServers":{"vid2context":{"command":"%s/node_modules/.bin/tsx","args":["%s/packages/mcp/src/bin.ts"]}}}\n' "$PWD" "$PWD"
  4. Keys are optional. To unlock more free rungs, add an env object to the entry, for example:

    json
    {
      "mcpServers": {
        "vid2context": {
          "command": "npx",
          "args": [
            "-y",
            "vid2context-mcp"
          ],
          "env": {
            "GROQ_API_KEY": "your-key"
          }
        }
      }
    }
  5. Quit and reopen Claude Desktop so it starts the server.

Check that it works#

The tools menu in a new chat lists vid2context with its tools.

Try it#

Ask Claude Desktop:

Get the transcript of https://www.youtube.com/watch?v=jNQXAC9IVRw and summarize it with timestamps.

The get_transcript tool hands the agent a short header and the timed transcript. This is the real text it returns for "Me at the zoo", rendered by vid2context from a recorded run:

text
Title: Me at the zoo
Channel: jawed
URL: https://www.youtube.com/watch?v=jNQXAC9IVRw
Duration: 0:19
Transcript: creator-captions, en (youtube-captions)
Chapters: 0:00 Intro | 0:05 The cool thing | 0:17 End

[0:01] All right, so here we are, in front of the elephants
[0:05] the cool thing about these guys is that they have really...
[0:07] really really long trunks
[0:12] and that's cool
[0:14] (baaaaaaaaaaahhh!!)
[0:16] and that's pretty much all there is to say

The agent can now cite moments by timestamp. For long videos it can call get_outline first and read only the sections it needs.

Optional keys#

Every key is optional. Each one unlocks more free rungs, which run before the keyless ones:

  • GROQ_API_KEY: Groq free tier: fast Whisper speech-to-text and LLMs
  • GEMINI_API_KEY: Google AI Studio free tier LLMs
  • OPENAI_API_KEY: OpenAI, used last and only when set
  • OLLAMA_BASE_URL: Local Ollama server, for example http://localhost:11434/v1
  • LOCAL_STT_BASE_URL: Local OpenAI-compatible speech-to-text server, for example http://localhost:8000/v1

The full list, with every model override, is in Providers and rungs.