Local files, direct media URLs and streams#

vid2context reads more than platform links: recordings on disk, direct links to .mp3 or .mp4 files, and HLS or DASH streams. This guide covers what works, how subtitles that come with a file are used, and how ffmpeg is found.

npm run cli -- ./talk.mp4
npm run cli -- ~/Recordings/interview.m4a --timestamps
npm run cli -- https://upload.wikimedia.org/wikipedia/commons/5/5a/Discovery_-_Nice_to_be_in_orbit.mp3
npm run cli -- https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8

Supported inputs#

Ref ids: local files are file:<sha256 of path, size and mtime>, so editing the file gives a new id and never serves a stale cache. URLs are media:<sha256 of the URL without its fragment>.

What you get#

For local files with ffmpeg installed, metadata comes from the container: the title tag (or the file name), artist (as the channel), date, comment, duration, container chapters, and the spoken language when the audio stream is tagged. extra.streams lists the streams.

Audio for speech-to-text:

• Video files: the first audio track is extracted as 16 kHz mono mp3 at 32 kbps, about 14 MB per hour. That keeps roughly 1.7 hours under the 25 MB upload limit of Groq and OpenAI; longer audio is split by the long-audio orchestrator.
• Audio files up to 25 MB in a format speech-to-text accepts (mp3 m4a wav flac ogg opus) go through untouched. Larger or other formats are re-encoded like video.

Subtitles that come with a file#

Before transcribing, media-file-captions looks for subtitles. When it finds usable ones, the transcript is creator-captions and no speech-to-text runs (check attempts: there is no audio or stt stage).

Sidecar files sit next to the media and share its name:

Modifiers sdh, cc and default are allowed in the name. Parsing handles a UTF-8 BOM, CRLF line endings, missing cue numbers, cue settings, multi-line cues, MM:SS timings, HTML-style tags, ASS override codes and WebVTT voice tags (kept as the speaker).

Embedded subtitle streams (mkv subrip or ass, mp4 mov_text, webvtt) are read with ffmpeg when there is no sidecar. Image-based subtitles (DVD, Blu-ray PGS) hold pictures, not text, and are skipped.

Order: sidecars before embedded streams, untagged sidecars first. With --lang xx, tracks in that language come first, then untagged ones; if only other languages exist, speech-to-text runs instead. The transcript's model field names the track used, for example sidecar:talk.en.srt or embedded:2.

ffmpeg#

@vid2context/ffmpeg finds ffmpeg and ffprobe in this order:

1. FFMPEG_PATH and FFPROBE_PATH. An explicit path that does not exist is an error, never silently replaced. ffprobe next to FFMPEG_PATH is found automatically.
2. PATH.
3. ~/.local/bin.
4. The managed copy in .vid2context/bin/ (or $VID2CONTEXT_HOME/bin).
5. A managed download, only when a video or stream actually needs ffmpeg, and only if VID2CONTEXT_FFMPEG_DOWNLOAD is not 0. Reading metadata never triggers it.

Without ffmpeg, audio files still work (raw bytes, filesystem metadata) and sidecar subtitles still work. Video files and streams fail with a hint to install ffmpeg.

Managed download#

On Linux x64 and arm64, vid2context downloads the newest release-branch static build from BtbN/FFmpeg-Builds (about 150 MB), checks it against the release's checksums.sha256, and installs ffmpeg and ffprobe into .vid2context/bin/ with a provenance file ffmpeg.json. A lock in .vid2context/locks/ makes parallel processes wait for one download instead of each starting their own.

These builds are GPL-licensed (they include x264 and x265). If that matters for how you distribute vid2context, install ffmpeg yourself and set VID2CONTEXT_FFMPEG_DOWNLOAD=0.

There is no managed download on macOS or Windows: no source with a published checksum was found (evermeet.cx, the usual macOS build, publishes GPG signatures only). Install with brew install ffmpeg or winget install ffmpeg.

Static builds and network input#

Some static builds, including the johnvansickle.com builds, crash (SIGSEGV) when they open an http(s) URL, because their statically linked C library cannot resolve DNS unless nscd is running. Local files are unaffected. Streams need ffmpeg to fetch the playlist itself, so with such a build, install ffmpeg from your package manager or point FFMPEG_PATH and FFPROBE_PATH at another build (the managed BtbN build works). vid2context reports this crash with that hint.

Safety#

ffmpeg runs without a shell, with arguments as an array. Inputs that start with - are refused, relative paths are made absolute (so concat: and other protocol names are never read from a path), every run has a timeout, an aborted request kills the process, and temp files are removed when done. For URL inputs ffmpeg may only use the http, https, tls, tcp and crypto protocols, so a hostile playlist cannot make it read local files.

Size limits#

The URL cap is checked against content-length (from HEAD, or a one-byte ranged GET when HEAD is refused) before downloading, and again while streaming, so a server that lies about the size is still stopped. Responses that are clearly not media (an HTML page) are refused with a hint to link the file itself.

Direct video URLs#

On Node, a video URL (.mp4, .webm, ...) is downloaded through the same capped fetch into a temp file, and ffmpeg extracts the audio locally. On Workers and in the browser there is no ffmpeg, so only video files up to 25 MB are accepted and sent to speech-to-text as they are.

HLS and DASH#

media-stream fetches the playlist for metadata: total duration (segment durations for HLS, mediaPresentationDuration for DASH), live or on-demand, variant count, audio renditions and subtitle languages. For a multivariant HLS playlist, audio comes from the default audio rendition's own playlist when it has one, otherwise from the smallest variant (an audio-only variant when there is one), so ffmpeg fetches as little as possible. Live streams are refused; for live transcription see watch_live in providers.md.

Verified live on 2026-11-14: Apple's public bipbop example (30 minutes of audio extracted in 16 seconds) and the Tears of Steel HLS demo (CC BY 3.0) transcribed through the keyless OVH rung.

For other packages#

@vid2context/ffmpeg is Node only. Never import it from core or from a provider that declares the workers or browser runtime.