Dojo Video Harness

A local, English-only open-source video editor with a small TypeScript editing agent. It shares DojoClip’s Next.js App Router, React 18, Tailwind 3, and editor components, with a simpler local host. Bring your own LLM and transcription providers. Edit the same timeline by hand or through chat, see changes directly, and export through Dojo Video Engine in your browser.

The agent is intentionally understandable: a model adapter, typed function calls, a tool registry, Markdown skills, bounded execution, and explicit user decisions. The difficult video-specific rules live in the tested document and command kernel. The shared loop 0.1.3 supports durable checkpoints and deterministic context pruning; the local host owns which interrupted boundaries can safely resume.

Browser-test screenshot with synthetic model responses.

Run locally

Requirements: Git, Node 20.19+ (Node 22 recommended), npm, and a desktop browser with WebCodecs support (Chromium is the tested path). No database, Redis, DojoClip account, or cloud worker is needed. FFmpeg is required only for the browser acceptance test, not the app.

git clone https://github.com/PansaLegrand/dojo-video-harness.git
cd dojo-video-harness
npm run launch

launch installs locked dependencies when missing, creates .env without overwriting an existing file, checks the local environment, and starts the editor. It opens the browser automatically from an interactive terminal. Otherwise open http://127.0.0.1:4310. The engine is installed automatically from the public @dojo-video-engine/* npm packages, pinned to 1.2.3; no separate engine clone or pnpm installation is needed. Node 22 is pinned in .nvmrc.

Create a project and import media. Manual editing, subtitle import, preview, motion graphics, and MP4 export work without API keys. For chat, configure Vercel AI Gateway or a GPT-6-series provider with a compatible Responses API. In Provider settings, enter its base URL, model ID, and API key using the configuration below. Speech recognition has its own provider settings. Saves apply to new requests immediately; running jobs keep their original configuration. Saved keys are never returned to the browser.

For later starts, use npm run dev. Other useful commands:

npm run setup        # Install/check without starting; preserves .env
npm run setup -- --install  # Reinstall locked dependencies
npm run doctor       # Check Node, local storage, and port availability
npm run dev -- --no-open    # Start without opening the browser

After pulling repository updates, run npm run setup -- --install before starting again. This installs the updated lockfile and preserves your .env and local project data.

The setup and launcher work in PowerShell as well as macOS/Linux terminals. The startup CI runs on Windows, macOS, and Linux. If the port is occupied, the launcher explains how to open the existing editor or change PORT in .env. It does not silently switch addresses, because imported browser media belongs to its origin.

Advanced configuration can still use .env (copied from .env.example). Settings saved in the app override its provider values and are stored in the private local data directory as provider-settings.json. Keep that directory private and out of Git when changing DATA_DIR.

DojoClip is the reference for shared editor and chat UI. The open-source engine and harness are the starting point for rendering and agent runtime changes; DojoClip consumes them through commercial adapters. Local persistence, provider keys, and usage copy stay in this harness's host adapters. See synchronization and readiness and syncing the product UI.

The agent and caption translation support GPT models through the Responses API. Vercel AI Gateway is the documented, live-tested example for GPT-6:

LLM_BASE_URL=https://ai-gateway.vercel.sh/v1
LLM_MODEL=openai/gpt-6-sol
LLM_API_KEY=your-vercel-ai-gateway-key

Other GPT-6-series providers can use the same adapter if their /responses endpoint supports the harness's function calls and tool-result continuation. Set the provider's base URL (without /responses), exact model ID, and matching key. Model IDs must use the gpt-* or openai/gpt-* form; gpt-6-sol is the default when no model is configured. GPT-6 availability alone does not establish API compatibility; validate another endpoint with the optional live check below. The harness does not silently switch providers or retry paid requests.

Use the key for the endpoint you configure. The model name is sent unchanged; there is no automatic model switching or retry. Non-GPT model names are rejected before a request. Other model families and Chat Completions are outside the supported scope. Speech recognition supports ElevenLabs Scribe and OpenAI-compatible word-timestamp endpoints with its own key; configuring chat does not configure ASR. Choose Use ElevenLabs in Provider settings to select https://api.elevenlabs.io/v1 and scribe_v1, then enter your own ElevenLabs key.

ASR must provide genuine word timestamps (verbose_json for compatible endpoints); segment-only output is rejected rather than converted into invented word timings. This adapter caps transcription uploads at 25 MiB. Choose a smaller audio source when needed.

Saved keys stay in the local data directory and server process. .env, project data, uploaded sources, and exported videos are ignored by Git. Imported originals first live in browser IndexedDB. Choosing transcription explicitly copies the source to the local host and sends it to the configured ASR provider. Choosing chat sends project context; preview pixels are included only when you check the preview-image option. Provider usage is charged by your provider. The editor and rendering run locally; configured remote GPT and ASR providers receive only the context or media needed for the selected operation.

To run the built application:

npm run build
npm start

What is implemented

• The extracted DojoClip manual editor: multi-track timeline, selection, trim, split, text/graphics, captions, styles, transitions, linked groups, source ranges, quiet-section analysis, undo/redo, and browser recovery.
• Dojo Video Engine preview, local proxies within admission limits, and browser MP4 export. The pinned engine packages include the reconciled DojoClip correctness patches.
• Local project persistence with serialized transactions, atomic file replacement, revisions, conflict detection, operation receipts, and history.
• A framework-independent agent loop using real function calls. Inspect a project, read a skill, search transcript evidence, propose an edit, or ask a question.
• New editor runs use local-editor-v5 with paginated asset/cue readers and the completion-preflight profile: freeze a typed checklist, review the complete in-memory candidate, repair failed checks, then propose exactly the reviewed commands. Deterministic receipts bind the snapshot, checklist, commands and preview; the host checks outcomes again after rebase.
• Durable local model/tool checkpoints, with cumulative budgets and frozen request, source, tool/skill and provider bindings. A restart can continue a saved model response or completed tool boundary without requesting that response again. Uncertain model or external-tool submissions stop for manual reconciliation.
• New runs reserve serialized UTF-8 space for the complete model reply before submission. Compact checkpoint storage retains original instructions and a pending reply once; accepted 0.1.1 runs retain their original profile and budget.
• Desktop chat resizes from its left edge, up to half the window width; the left panel toggle stays at the bottom of the tool rail. Narrow screens retain the chat drawer.
• Instruct and edit: ordinary local chat edits apply automatically, with selective Undo and optional before/after comparison. Structured questions resolve missing information; cancellation and the product’s quick-edit controls remain available. Tool events remain available through the local API.
• Provider-backed captions and translation, with persisted source-bound receipts.
• Speech-dependent chat pauses for explicit source/provider consent when genuine transcript evidence is missing, then resumes the original instruction after the source-bound receipt is available. Later styling, translation and graphic steps can use that evidence in the same bounded model/tool run.
• Readable title, lower-third and supplied-text caption preparation with explicit bounds and responsive defaults; chat applies them without a confirmation click. See layout checks and bounded real-footage acceptance.
• Native countdowns and animated text can be separate, editable timeline clips before or after footage. The