OpenClaw Voice Call — Realtime Edition

Give your AI assistant a phone.

This OpenClaw plugin lets your assistant place and receive real phone calls and hold natural, full-duplex voice conversations — powered by Twilio Programmable Voice and the OpenAI Realtime API. It books your table, checks the store's hours, navigates the phone menu, gets a human on the line, wraps up politely, hangs up on its own, and reports back with a structured outcome, an AI-written summary, and the full transcript.

You:        "Call Luigi's and book a table for two on Friday at 7."
Assistant:  *dials, talks past the IVR, negotiates with the host, confirms details*
Assistant:  "Booked — table for 2, Friday 7pm, under your name.
             They said parties over 15 minutes late lose the table."

Why this exists

Every "AI agent" can send an email. Almost none of them can call the dry cleaner. The real world still runs on phone calls — restaurants, doctors' offices, contractors, that one store whose website hasn't been updated since 2019. This plugin closes that gap: your assistant gets a phone number, a voice, ears, a keypad, and the judgment to end the call when the job is done.

Features

• Full conversation mode — Twilio Media Streams bridged to the OpenAI Realtime API (speech-to-speech, GA protocol). Sub-second turnaround, natural barge-in when the other party interrupts.
• In-call tools the voice AI uses autonomously:
  • press_phone_keys — synthesized DTMF touch-tones for navigating IVR menus ("press 2 for reservations")
  • report_call_outcome — structured result capture (status + every fact gathered: times, prices, confirmation numbers)
  • end_call — graceful hangup: speaks a closing line, waits for it to actually play out on the line (Twilio mark echo), then disconnects. No clipped goodbyes, no lingering dead air.
  • ask_assistant (optional) — the killer feature for scheduling: mid-call, the voice AI relays a question to your OpenClaw agent ("is Wednesday 2pm free?") and speaks the answer. Your agent answers with its full toolset — calendar, search, smart home, anything — so the phone persona stays thin and your agent stays the brain.
  • check_calendar (optional) — zero-dependency alternative: free/busy straight from a secret iCal feed URL (Google Calendar, iCloud, Outlook, Fastmail). Only busy windows are exposed — never event titles or details.
  • check_home / control_home (optional) — direct Home Assistant backend: read device status ("are the doors locked?") and, when enabled, control devices ("turn off the porch light") in ~1 second, bypassing the agent. Offered only on verified owner/trusted calls.
  • ask_owner (optional) — mid-call text escalation: the AI messages you ("7pm is taken — is 8pm OK?"), you reply by text, and your answer flows back into the live call while the other party holds. Far lighter than a transfer.
  • transfer_to_owner (optional) — when the other party genuinely needs a human (payment details, authorization), the AI announces the handoff and transfers the call to your real phone.
• Transcripts & summaries — every call is finalized into a Markdown transcript with an AI-generated summary and the reported outcome. Retrievable after the call via the get_transcript tool action.
• Post-call reports (optional) — when a call ends, your OpenClaw agent is briefed automatically: it messages you the result and performs obvious follow-ups (adds the confirmed booking to your calendar, notes a needed retry). Calls stop being fire-and-forget.
• Answering-machine detection (optional) — Twilio async AMD tells the plugin definitively when voicemail picked up; policy decides: leave a concise message after the beep (default), hang up, or continue as normal.
• Semantic turn detection (optional) — semantic_vad understands when a speaker has finished a thought rather than just paused, for fewer interruptions and more natural conversations.
• Call screening awareness — handles Google Call Screen and voicemail gatekeepers by identifying itself with a configurable identity phrase.
• Goal-directed calls — pass talking_points and call_party (first-party vs third-party) and the AI stays on task: cover the points, collect the answers, confirm out loud, wrap up.
• Inbound calls (optional) — allowlist-gated with layered identity verification: SHAKEN/STIR attestation (the carrier cryptographically vouches for the caller ID, defeating spoofing) and an optional spoken access phrase checked by a tool — the phrase never enters the model prompt.
• Device profiles — per-caller policies: response length, forbidden actions, extra instructions.
• Hardened by default — Twilio webhook signature verification, per-call stream auth tokens, pre-auth connection throttling, SSRF-guarded provider API calls, call-duration safety caps, stale-call reaping.
• Providers — Twilio (recommended, full realtime conversation mode), plus Telnyx / Plivo / mock for the legacy TTS+STT pipeline.

Use cases

• Reservations & appointments — restaurants, salons, dentists: "book me the earliest slot Thursday afternoon." With the assistant bridge or calendar tool enabled, the AI checks your real availability before agreeing to a time.
• Information gathering — opening hours, stock checks, price quotes, "is the kitchen still open?"
• Errand triage — call the pharmacy about a refill, the contractor about a quote, the venue about parking.
• Notifications with a human touch — a call that speaks a message and confirms it was heard, not just a text that might be missed.
• Reaching you — your assistant can call your own phone when something needs a real-time decision, and you can just talk to it.
• Inbound assistant line — allowlisted callers can ring your assistant directly and converse with it.

How it works

 Phone network                Your machine                        OpenAI
┌─────────────┐   webhooks   ┌──────────────────────┐
│   Twilio    │ ───────────► │  Plugin webhook       │
│ Programmable│              │  server (HTTP)        │
│    Voice    │ ◄─────────── │  · TwiML <Connect>    │
│             │   TwiML      │    <Stream>           │
│             │              │                       │
│  Media      │  WebSocket   │  Media stream         │  WebSocket  ┌──────────┐
│  Streams    │ ◄══════════► │  handler (μ-law 8kHz) │ ◄═════════► │ Realtime │
└─────────────┘   audio      └──────────────────────┘    audio    │   API    │
                                                                   └──────────┘

The plugin runs an HTTP server inside the OpenClaw gateway. Twilio fetches TwiML from it and opens a bidirectional audio WebSocket; the plugin bridges that audio to an OpenAI Realtime session which does STT, reasoning, tool calls, and TTS in one loop.

Requirements

• OpenClaw ≥ 2026.6 (the plugin loads as a path plugin and runs inside the gateway)
• Twilio account with a voice-capable phone number (~$1.15/mo + ~$0.014/min)
• OpenAI API key with Realtime API access (gpt-realtime-2.1 recommended)
• A public HTTPS URL reaching the plugin's webhook port (Cloudflare Tunnel, ngrok, or Tailscale Funnel — walkthrough below)

Installation

1. Get the plugin

mkdir -p ~/.openclaw/plugins-src
git clone https://github.com/TristanBrotherton/openclaw-voice-call-realtime.git \
  ~/.openclaw/plugins-src/voice-call-realtime
cd ~/.openclaw/plugins-src/voice-call-realtime
npm install --omit=dev

2. Set up Twilio (detailed)

1. Create an account at twilio.com. Trial accounts work for testing but can only call verified numbers and play a trial notice — upgrade (add billing) for real use.
2. Buy a voice-capable phone number: Console → Phone Numbers → Manage → Buy a number. Check the Voice capability box. Pick a local number; note it in E.164 form (e.g. +15551234567).
3. Get your credentials: Console home page → Account Info panel:
   • Account SID — starts with AC…
   • Auth Token — click to reveal. Treat it like a password; it signs and verifies every webhook.
4. Geo permissions (only if calling outside your country): Console → Voice → Settings → Geo permissions — enable the destination countries.
5. Inbound calls (optional): Console → your phone number → Voice Configuration → set "A call comes in" to Webhook, URL = your public webhook URL (below), method POST. For outbound-only use you can skip this — the plugin passes TwiML URLs per call via the API.

3. Expose the webhook publicly

Twilio must reach the plugin's webhook server (default port 3336) over HTTPS. Cloudflare Tunnel (free, stable hostname) is recommended:

brew install cloudflared            # or apt/pacman equivalent
cloudflared tunnel login            # authorize against your Cloudflare-managed domain
cloudflared tunnel create voice
cloudflared tunnel route dns voice voice.yourdomain.com

Create ~/.cloudflared/config.yml:

tunnel: voice
credentials-file: /Users/you/.cloudflared/<tunnel-id>.json
ingress:
  - hostname: voice.yourdomain.com
    service: http://localhost:3336
  - service: http_status:404

Run it: cloudflared tunnel run voice

Make it survive reboots. A dead tunnel silently breaks all calls. On macOS, install a LaunchAgent (~/Library/LaunchAgents/com.you.cloudflared-voice.plist):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key><string>com.you.cloudflared-voice</string>
  <key>ProgramArguments</key>
  <array>
    <string>/opt/homebrew/bin/cloudflared</string>
    <string>tunnel</string><string>--config</string>
    <string>/Users/you/.cloudflared/config.yml</string>
    <string>run</string><string>voice</string>
  </array>
  <key>Ru