HTTP API contract#

One Hono app (packages/server/src/app.ts) serves every runtime. All routes live under /api/v1. Bodies are JSON unless a route says otherwise. Data types are the core types in packages/core/src/types.ts and packages/core/src/storage.ts, serialized as JSON.

Sections: conventions, implemented routes, planned routes. Planned routes are a contract: build them with exactly these paths and shapes, or change this page in the same commit.

Conventions#

Errors#

Every error response has this body:

{
  "error": { "code": "not-found", "message": "YouTube xxxxxxxxxxx: ERROR (This video is unavailable)", "hint": "Check the video id or URL; the video may have been removed." },
  "attempts": [{ "id": "youtube", "stage": "metadata", "outcome": "error", "reason": "not-found", "detail": "..." }]
}

code is a core ErrorCode, internal for unexpected errors (status 500, no detail leaked), or one of the server's own codes about the caller's credentials: unauthorized (401, missing or wrong API key, with WWW-Authenticate: Bearer) and forbidden (403, the key lacks the scope). hint is present when there is a recovery step. attempts lists every provider and rung tried (empty when none ran). rate-limited responses carry Retry-After in seconds.

The map lives in STATUS_BY_CODE in packages/server/src/errors.ts (re-exported from app.ts). Add new codes there and here together.

Common query parameters#

Headers#

• X-Request-Id: sent back on every response. A client value of up to 128 characters from A-Za-z0-9._:- is kept; anything else is replaced by a UUID. Access logs carry it.
• ETag on GET /context and GET /library/:id (SHA-256 of the body). If-None-Match with a matching tag answers 304 with no body.
• X-Robots-Tag: noindex on every /api/* response, so JSON never competes with the docs in search results.
• X-RateLimit-Limit and X-RateLimit-Remaining on rate-limited routes; Retry-After on 429.
• X-Data-Attribution when a response used SponsorBlock data (skip=).

Secrets#

Responses never contain env var values. Health output names the variables a rung needs (requires), never their contents.

Implemented routes#

GET /api/v1/health#

{
  "ok": true,
  "name": "vid2context",
  "version": "0.1.0",
  "runtime": "node",
  "formats": [{ "id": "text", "description": "...", "mimeType": "text/plain; charset=utf-8" }],
  "providers": [
    { "id": "youtube", "kind": "source", "name": "YouTube", "enabled": true },
    {
      "id": "stt-openai-compatible", "kind": "stt", "name": "OpenAI-compatible speech-to-text",
      "rungs": [{ "id": "stt/ovh/whisper-large-v3-turbo", "service": "ovh", "model": "whisper-large-v3-turbo", "tier": "keyless", "enabled": true, "requires": [] }]
    }
  ]
}

enabled on a rung means its credentials are present and its tier is allowed (keyless rungs report false when VID2CONTEXT_KEYLESS=0).

GET /api/v1/context?url=&mode=&lang=#

Returns the full VideoContext: metadata, transcript (timed segments), optional visual and comments, and attempts.

GET /api/v1/transcript?url=&format=&timestamps=&mode=&lang=&skip=&translate=#

Returns the transcript rendered by the chosen formatter (default text) with that formatter's mimeType as Content-Type.

• skip=sponsor,selfpromo (or skip=1 for the default categories) removes SponsorBlock segments from a copy of the transcript through @vid2context/skips; the stored transcript stays whole, and the response carries X-Data-Attribution.
• translate=1 needs item 35's translation layer, which is not installed: the server answers bad-request with a hint (pass lang=xx for a track the platform already has).

Routes implemented by item 46#

Shapes are those in the planned tables below; these notes add what the tables leave open. Every route that takes url also takes upload:<sha256>.

The library is the registry's enabled storage backend, opened once per process (or per Worker isolate), else a process-wide MemoryStorage. Every route that loads a context reads and writes it, except uploads, which are never saved there.

Identical concurrent requests that load a context (same input, mode, language, comments and refresh) share one pipeline run. The shared run has its own timeout, so one client disconnecting does not fail the others.

GET /api/v1/models?kind=#

{ models: ModelInfo[], defaultPolicy: Policy } (ModelInfo as in docs/prompts/SHARED-CONTRACTS.md section 3). Every rung the registry knows, enabled or not; kind is one of llm, stt, embed, video, visual. label is the model id after its last /. capabilities gains stream for LLM rungs that stream natively. enabled follows the same rule as health (credentials present, keyless allowed). requires lists env var names, never values. defaultPolicy is VID2CONTEXT_POLICY or free.

POST /api/v1/uploads#

Multipart form with a field file. Answers 201 { input: "upload:<sha256>", filename, bytes, mimeType, expiresAt }.

• Size cap VID2CONTEXT_MAX_UPLOAD_MB (the server's default is 100, the Workers free-plan request limit): larger files get too-large (413). Empty files and bodies that are not multipart/form-data get bad-request.
• The id is the SHA-256 of the bytes, so the same file uploaded twice is one upload (its expiry is extended).
• Kept for VID2CONTEXT_UPLOAD_TTL_MIN minutes (default 60). When the storage backend can hold blobs (the Cloudflare backend with an R2 bucket bound), the bytes go to R2 under blobs/uploads/<sha256> and the details and transcripts to the KvStore (uploads: keys) with the same TTL, so every isolate sees them. Otherwise they stay in the process's memory (at most four uploads at the size cap at once, oldest dropped first) and are lost on restart.
• upload:<sha256> is resolved by the server, not the registry: transcribed with core transcribe() from an AudioInput of kind bytes, with minimal metadata from the filename (title without extension, extra: { filename, bytes, mimeType }). Modes captions and video give unsupported-input. The transcript is kept next to the upload, never in the library. An expired or unknown id is not-found; a malformed one bad-request.
• Large video files should be reduced to audio before uploading (the web app does this in the browser): speech-to-text services cap uploads at about 25 MB, so compressed mono audio (about 30 MB per hour at 64 kbit/s) goes much further.

GET /api/v1/bundle?url=&formats=&comments=&maxTokens= and POST /api/v1/bundle#

application/zip built by core buildBundle (layout in docs/prompts/SHARED-CONTRACTS.md section 2), streamed, with Content-Disposition: attachment; filename="<root>.zip". formats is a comma-separated list (GET) or array (POST) of formatter ids; context.md is always written. POST takes { inputs?, collectionId?, formats?, comments?, maxTokens?, mode?, lang? } with the same 100-video cap as POST /pack.

GET /api/v1/openapi.json#

OpenAPI 3.1 for every route the app serves, generated from the routes Hono actually has: each route module registers a description next to its handler (packages/server/src/openapi.ts), and mounted sub-app routes without a description appear with a generic entry tagged by package. A package may publish descriptions for its routes by exporting an OpenAPI paths object (relative to its mount prefix); the server passes it in when mounting. /health is marked open (security: []).

Mounted sub-apps#

Mounted by createApp (packages/server/src/app.ts, MOUNTED_SUB_APPS). They share the server's Ctx per request, its storage and its KvStore, and pass through the same middleware under /api/*.

Not mounted yet, because the package does not export a factory: insights (32, /briefs, /extract), translation (35), chapters (36), the hosted-mode accounts middleware and routes (58), bots (56), live (61) and duplicates (62).

Planned routes#

The Owner column names the work item (see docs/ROADMAP.md) that implements the route.

Ownership rule for parallel work: only item 46 edits packages/server. An item whose own package backs routes (32 to 36, 40, 41, 42) exports a Hono sub-app factory from src/http.ts in that package, implementing exactly the paths below relative to /api/v1:

// packages/library/src/http.ts
import type { Ctx, Registry, Storage } from '@vid2context/core';
import { Hono } from 'hono';
export interface RouteDeps { registry: Registry; makeCtx: (req: Request) => Ctx; storage: (ctx: Ctx) => Promise<Storage> }
export function libraryRoutes(deps: RouteDeps): Hono { /* GET /collections, POST /collections, ... */ }

Item 46 mounts every sub-app that exists when it runs (app.route('/api/v1', libraryRoutes(deps))). A sub-app that lands later is mounted with a one-line additive change to packages/server/src/app.ts. Errors thrown inside a sub-app propagate to the main app's onError, so they get the standard body and status. Routes whose owner is 46 are built by 46 directly from core and provider APIs.

Videos#

Library and collections#

Collection types (@vid2context/library): Schedule is { every: 'hourly' | 'daily' | 'weekly' | { minutes: number }, at?: 'HH:MM' (UTC), enabled }. RefreshSummary is { collectionId, added: VideoSummary[], skipped: number, failed: { input, code, message, hint?, cause? }[], remaining: number, checkedAt }. Ingest and refresh jobs report progress { done, total, message } and one item event { id, outcome: 'ok' | 'error', title?, error? } per video. See guides/collections.md.

Intelligence#

Citation is { videoId, title, startSec, endSec?, url?, quote? }, where url is a timestamp link from timestampUrl. /ask and /chat add n, the [n] in the answer the citation backs; startSec is narrowed to the segment that best supports the citing sentence.

POST /chat (and POST /ask with stream: true)#

Multi-turn, model-selectable answers grounded in videos. Implemented by askRoutes in packages/ask/src/http.ts; see guides/ask.md.

interface ChatApiRequest {
  messages: { role: 'user' | 'assistant'; content: string }[]; // oldest first, ending with the user's question
  /** Omit for Auto (the server's policy, free-first). */
  model?: { rung: string; strict?: boolean } | { policy: 'free' | 'quality' | 'local-only' };
  /** What to ground the answer in: URLs, ids like youtube:abc, or upload:<sha256>, and/or a collection. */
  context?: { inputs?: string[]; collectionId?: string };
  maxTokens?: number;
  temperature?: number;
}

model.rung with strict: false (the default) starts with that rung, then falls back along the chain; strict: true means that rung or an error. /ask takes the same model.

Events, in order (event: name, data: one JSON object). A comment line (: keep-alive) is sent every 15 seconds; a client disconnect aborts the upstream call.

The answer cites sources as [n], matching meta.sources[].n. A number that matches no source is removed from the streamed text and from done.text, and listed in done.unverified. Request errors (bad body, unknown collection, a video that cannot be had) are answered before streaming with the standard error body and status; failures after streaming starts (every rung failed, a broken stream) arrive as an error event.

Jobs#

Long work (collection ingest, briefs, long audio) runs as a job. Owner: 42.

Stream details (implemented in packages/jobs/src/sse.ts):

• Event ids are a per-job sequence starting at 1. A client that reconnects with Last-Event-ID: n receives only events after n.
• A fresh connection first replays the state: the item events so far, then the latest progress (or the final done or error of a finished job), then live events. A comment line : heartbeat is sent every 15 seconds. The stream closes after done or error.
• item may also carry title and, for failures, error: { code, message, hint? }; outcome is ok, error or skipped.
• A canceled job ends with error { code: 'canceled', ... }, and its Job.error has the same value.
• Built-in kinds: context (input: one URL), items (input: a playlist, channel or feed; options limit, concurrency), bundle (input: a URL or an array of URLs). See docs/guides/jobs.md.

interface Job {
  id: string;
  kind: string; // 'ingest', 'refresh', 'brief', 'extract', 'transcribe-long', ...
  status: 'queued' | 'running' | 'succeeded' | 'failed' | 'canceled';
  progress?: { done: number; total?: number; message?: string };
  result?: unknown;
  error?: { code: string; message: string; hint?: string };
  createdAt: string; // ISO 8601
  updatedAt: string;
}

Platform#

Auth and limits (owner: 45 and 46, implemented by 46 for /api/*)#

• API keys: when VID2CONTEXT_API_KEYS is set (comma-separated), every /api/* route except /health requires Authorization: Bearer <key>; otherwise the server is open and the Node entry binds to 127.0.0.1 by default. Each entry is a plain key (full access) or a hashed entry name:scope:sha256hex, where the hex is the SHA-256 of the key and the scope is read, write or admin. read allows GET and the POST routes that only compute an answer (/pack, /bundle, /ask, /chat, /research, /duplicates, /exports/obsidian); write also allows every other POST, PUT, PATCH and DELETE; admin allows everything. Keys are compared by SHA-256 in constant time. Missing or wrong keys get 401 unauthorized, a key without the scope 403 forbidden. Logs name the key (key-1 for the first plain key, or the hashed entry's name), never the key itself. /mcp and the OAuth paths use mcp-http's own credentials.
• Rate limiting: a token bucket per key (or per client IP in open mode) in a KvStore (ratelimit: keys; the storage's own KV when it has one, else process memory). VID2CONTEXT_RATE_LIMIT requests per minute (default 120, 0 off) with bursts of VID2CONTEXT_RATE_BURST (default half the rate). Over the limit: 429 with Retry-After and the standard error body (code: rate-limited). /health and CORS preflights are not limited. The client IP is the socket address on Node and CF-Connecting-IP on Workers; X-Forwarded-For and X-Real-IP count only with VID2CONTEXT_TRUST_PROXY=1.
• CORS is off by default; VID2CONTEXT_CORS_ORIGINS lists allowed origins (https://vid2context.com, chrome-extension://<id>, or *). Preflights are answered before auth. Allowed request headers: Authorization, Content-Type, X-Request-Id, If-None-Match; exposed: X-Request-Id, Retry-After, ETag, Content-Disposition, X-Data-Attribution, X-RateLimit-*.
• Body limits: VID2CONTEXT_MAX_BODY_KB (default 1024) for JSON bodies, the upload cap for POST /uploads; larger bodies get 413 too-large.
• Timeout: a request that runs longer than VID2CONTEXT_REQUEST_TIMEOUT_MS (default 300000) is answered with 504 timeout, and its Ctx signal aborts the providers' work.
• Public mode: with VID2CONTEXT_PUBLIC=1, inputs that are local paths, file:// URLs or private addresses are refused with unsupported-input before any provider runs (route check plus the pipeline's own).
• Access logs: one JSON line per request on stderr: ts, method, path, status, ms, requestId, key (name), ip. VID2CONTEXT_ACCESS_LOG=0 turns them off.
• Node shutdown: on SIGTERM or SIGINT the server stops accepting connections, lets in-flight requests finish for up to VID2CONTEXT_SHUTDOWN_TIMEOUT_MS (default 10000), closes MCP sessions and the jobs queue, then exits.

Worker: site discovery (item 51)#

The Worker entry (packages/server/src/workers.ts) sends /api/* and, with mcp-http mounted, its paths to the app; everything else is a static asset, through serveSite in packages/server/src/discovery.ts:

• A GET or HEAD whose Accept ranks text/markdown above text/html gets the page's .md twin (/index.md for /, then <path>.md, then <path>/index.md) with Content-Type: text/markdown; charset=utf-8 and Vary: Accept. Without a twin, the HTML is served.
• HTML responses get Vary: Accept and Link: </llms.txt>; rel="alternate"; type="text/plain", </api/v1/openapi.json>; rel="service-desc"; type="application/openapi+json", </mcp>; rel="service" (the last only when mcp-http is mounted).
• www and http are redirected at the zone, not in code.

Wave 2 routes (prompts 51 to 62)#

Same conventions as above. Each package exports a Hono sub-app factory from its own src/http.ts; item 46, or the integration pass, mounts it under /api/v1.

Hosted mode (58): when VID2CONTEXT_HOSTED=1, the accounts middleware runs in front of every route except /health, /auth/* and /bots/*, enforcing sessions or API keys, per-account quotas and per-IP limits.