Give Claude the ability to watch and understand videos — Claude Code plugin with frame extraction and multimodal audio analysis
# Add to your Claude Code skills
git clone https://github.com/jordanrendric/claude-video-visionGuides for using mcp servers skills like claude-video-vision.
Last scanned: 5/12/2026
{
"issues": [],
"status": "PASSED",
"scannedAt": "2026-05-12T06:40:29.505Z",
"semgrepRan": false,
"npmAuditRan": true,
"pipAuditRan": true
}claude-video-vision is an open-source mcp servers skill for AI coding assistants such as Claude Code, Codex CLI, and ChatGPT, built by jordanrendric. Give Claude the ability to watch and understand videos — Claude Code plugin with frame extraction and multimodal audio analysis. It has 1,290 GitHub stars.
Yes. claude-video-vision passed SkillsLLM's automated security scan — a dependency vulnerability audit plus prompt-injection heuristics — with no high-severity issues. You can read the full report in the Security Report section on this page.
Clone the repository with "git clone https://github.com/jordanrendric/claude-video-vision" and add it to your Claude Code skills directory (see the Installation section above).
claude-video-vision is primarily written in TypeScript. It is open-source under jordanrendric on GitHub, so you can review or fork the full source.
Yes. SkillsLLM lists many other MCP Servers skills you can browse and compare side by side. Open the MCP Servers category from the badge at the top of this page, or use the Related Skills and comparison links further down to weigh claude-video-vision against similar tools.
No comments yet. Be the first to share your thoughts!
Top skills in this category by stars
⚠️ Third-Party Software Notice
This skill is third-party open-source software developed and hosted independently on GitHub. SkillsLLM is an informational directory and does not control or maintain the underlying repository.
Any security checks, ratings, or warnings displayed by SkillsLLM are automated and limited in scope. They do not constitute a security certification or guarantee that the software is safe, error-free, or free from malicious code, vulnerabilities, compromised dependencies, or prompt-injection risks.
Review the source code, permissions, dependencies, and configuration before installing or running any third-party skill. Use is at your own risk. To the maximum extent permitted by applicable law, SkillsLLM is not liable for losses arising from third-party software.
Give Claude the ability to watch and understand videos.
A Claude Code plugin that extracts frames via ffmpeg and processes audio via multiple backends (Gemini API, local Whisper, or OpenAI API). Claude receives frames as images and audio transcription with timestamps — the plugin is a perception layer, not an interpretation layer.
yt-dlp, preserving source metadata and captions for context/setup-video-vision walks you through configurationInside Claude Code, run these commands one at a time:
/plugin marketplace add https://github.com/jordanrendric/claude-video-vision
Then:
/plugin install claude-video-vision
The MCP server will auto-install via npx from npm on first use — no build step required.
Alternative: local development
git clone https://github.com/jordanrendric/claude-video-vision.git
claude --plugin-dir /path/to/claude-video-vision
Inside Claude Code, run the interactive wizard:
/claude-video-vision:setup-video-vision
It will walk you through backend selection, whisper configuration (if local), frame options, and dependency verification.
/watch-video path/to/video.mp4
/watch-video tutorial.mp4 "what language is used in this tutorial?"
/watch-video https://www.youtube.com/watch?v=... "summarize this video"
Just mention a video file or YouTube URL — Claude will detect it:
"analyze this video for me: ~/Downloads/demo.mp4"
"take a look at the first second of ~/videos/bug-report.mov"
"summarize this YouTube Short: https://www.youtube.com/shorts/..."
Claude adapts parameters automatically:
00:00:00 to 00:00:01| Backend | Audio processing | Cost | Setup |
|---|---|---|---|
| Gemini API | Native (speech + non-speech events) | Free tier: 1500 req/day | GEMINI_API_KEY env var |
| Local (Whisper) | whisper.cpp or Python openai-whisper |
Free, fully offline | brew install whisper-cpp + auto model download |
| OpenAI API | OpenAI Whisper API | Paid per usage | OPENAI_API_KEY env var |
All backends extract video frames via ffmpeg — Claude always has direct visual access.
┌───────────────────────────────────────────────────────┐
│ Claude Code (your session) │
│ │
│ /watch-video ──→ Skill: video-perception │
│ │ │
│ ▼ │
│ MCP tool: video_watch │
│ │ │
└────────────────────────┼──────────────────────────────┘
│
▼
┌────────────────────────────────────┐
│ MCP Server (Node.js) │
│ │
│ ┌──────────┐ ┌──────────────┐ │
│ │ ffmpeg │ │ Audio backend│ │
│ │ frames │ ║ │ (parallel) │ │
│ └──────────┘ └──────────────┘ │
│ │ │ │
└───────┼─────────────────┼──────────┘
▼ ▼
base64 images transcription
+ timestamps + audio events
│ │
└────────┬────────┘
▼
Claude receives both
brew install yt-dlp on macOS)brew install whisper-cpp (macOS) or equivalentThe plugin exposes 6 MCP tools:
video_watch — Extract frames + process audio (main tool)video_analyze — Analyze video structure with ffmpeg filters before extractionvideo_detail — Drill into specific cached or newly extracted momentsvideo_info — Get video metadata without processingvideo_configure — Change settingsvideo_setup — Check and guide dependency installation/watch-video <path> [question] — Analyze a video/setup-video-vision — Interactive configuration wizardSettings are stored in ~/.claude-video-vision/config.json:
{
"backend": "local",
"whisper_engine": "cpp",
"whisper_model": "auto",
"whisper_at": false,
"frame_mode": "images",
"frame_format": "jpeg",
"frame_resolution": 512,
"default_fps": "auto",
"max_frames": 100,
"frame_describer_model": "sonnet",
"enable_index": false,
"session_max_age_days": 7,
"downloads_max_age_days": 7
}
frame_format can be jpeg, png, or webp. jpeg remains the default for backwards compatibility; png is useful for screen recordings where text and sharp UI edges should stay lossless.
Whisper models auto-download to ~/.claude-video-vision/models/ on first use. Available: tiny, base, small, medium, large-v3-turbo, large-v3, auto (picks best for your RAM).
For YouTube URLs, the server uses this transcript order:
Audio results label provenance with transcription_source, for example youtube_subtitles or youtube_auto_captions, so Claude can treat manual subtitles as stronger evidence than auto-captions.
v1.0.0 — Initial release. Tested on macOS (Apple Silicon) with Local backend (whisper.cpp).
MIT — see LICENSE.
Jordan Vasconcelos