ccLoad

AI API gateway for Claude Code, Codex, Gemini, and OpenAI.

English | 简体中文

Smart routing | Automatic failover | Model-aware cooldown | Multi-URL scheduling | Protocol transforms | Live monitoring | Cost control

ccLoad removes the operational mess of running multiple AI API upstreams. It keeps Claude Code, Codex, Gemini, and OpenAI-compatible clients on one stable gateway, then handles upstream selection, failover, cooldown, protocol conversion, request visibility, and cost limits in the service instead of in every client script.

🤖 Built with Codex and GPT-5.6

During OpenAI Build Week, Codex powered by GPT-5.6 was the primary engineering agent used to:

• Trace request routing, failover, cooldown, protocol conversion, and dashboard flows across the Go backend and embedded web UI.
• Implement and review model-scoped cooldown handling for upstream 5xx, key-level 429, model-unavailable 404, and explicit model-retirement 410 failures without unnecessarily cooling an entire channel.
• Refine the model-status and call-statistics UI, update the English and Chinese documentation, and verify the result with focused Go tests, builds, and browser walkthroughs.
• Prepare the reproducible demo and Devpost submission while keeping architecture, security, and final-review decisions under human control.

GPT-5.6 is also integrated into the product itself: ccLoad exposes GPT-5.6 through OpenAI-compatible and Codex Responses endpoints, includes Sol, Terra, and Luna model presets, calculates their standard, priority, flex, cached-token, and long-context costs, and applies routing and model-scoped cooldown decisions to them like any other configured upstream model.

The repository's AGENTS.md and CLAUDE.md provide persistent engineering constraints so Codex works against the same KISS-first review and testing rules in every session.

🎯 What ccLoad Solves

Common failure modes when you run several AI API channels:

• Manual channel switching: Different keys, validity windows, quotas, and upstream URLs quickly become hard to manage.
• Rate limits and upstream failures: 429, 502, 504, expired keys, and overloaded providers should not stop the client workflow.
• Opaque request status: Without live request visibility, long streaming requests become guesswork.
• HTTP 200 with error content: Some upstreams return a successful HTTP status while the response body is an actual error.
• Cost drift: Shared gateways need per-channel and per-token limits, not spreadsheet accounting after the bill arrives.

ccLoad handles those cases with:

• Smart routing: High-priority channels are selected first; channels at the same priority use smooth weighted round-robin.
• Automatic failover: Failed keys, models, channels, and URLs are skipped according to the classified error scope.
• Model-aware cooldown: Structured model_cooldown responses, upstream HTTP 5xx failures, key-level 429 rate limits, model-unavailable 404 errors, and explicit model-retirement 410 errors all cool only the actual upstream model first; other models on the same channel remain available. The channel is promoted to cooldown only after every configured model or every enabled key is cooling.
• Multi-URL scheduling: A single channel can use multiple upstream URLs, weighted by observed latency and health.
• Per-URL protocol routing: Each URL can declare the upstream wire protocols it accepts. Explicit declarations route directly; an empty declaration tries the client protocol first and caches the working fallback.
• Responses WebSocket bridging: Authenticated Codex clients can keep a downstream WebSocket while each candidate uses native Codex WebSocket or the existing HTTP/SSE transport.
• Live monitoring: Active requests, logs, token usage, TTFB, cost, and upstream details are visible in the web dashboard.
• Soft-error detection: HTTP 200 responses that are actually errors trigger the same failover path as regular upstream failures. Common cases include:
  • JSON responses containing {"error": {...}} structure
  • Responses with type field set to "error"
  • Explicit rate limits in SSE error events (rate_limit_exceeded / too_many_requests) are handled as 429
  • Plain text messages like "当前模型负载过高" / "Current model load too high" (load warnings)

✨ Key Features

• 🚀 High-Performance Architecture - Gin framework, 1000+ concurrent connections, high-performance caching
• 🧮 Local Token Counting - API-compliant local token estimation, <5ms response, 93%+ accuracy, supports large-scale tool scenarios
• 🎯 Smart Error Classification - Distinguishes Key/Model/Channel/Client errors, soft error detection (200 masquerading as error), SSE rate-limit errors as 429, 1308 quota handling
• 🔀 Smart Routing - Priority + smooth weighted round-robin channel selection, pre-filters cooled channels, multi-key load balancing, health-based dynamic sorting (confidence factor prevents small sample over-penalization)
• 🛡️ Failover - Key, model, and channel failures share one exponential-backoff policy; explicit upstream reset deadlines take priority, and model-scoped failures switch channels without cooling the whole channel
• 🔒 Race-Safe - Key selector race condition protection, startup config validation, automatic resource cleanup
• 📊 Real-time Monitoring - Built-in trend analysis, logging, and stats dashboard, Token usage stats with time range selection and per-token classification, runtime status panel with process metrics (CPU, RSS, GC)
• 🎯 Transparent Proxy - Supports Claude Code, Codex, Gemini, and OpenAI compatible APIs with smart auth detection
• 🔑 OAuth Channels - Codex (ChatGPT), Anthropic (Claude), Antigravity, and xAI OAuth credentials with automatic refresh where supported; Codex personal access token (PAT) authorization; Z.ai Coding Plan (ZCode) browser authorization or API-key import; Cursor user API-key import; and Zed native sign-in (trial bound to a real Zed installation's system_id), with batch quota refresh, invalid-credential cleanup, and auto-disable for permanently rejected credentials
• 📅 OAuth Quota Cost Tracking - Per-credential weekly/monthly standard-cost accumulation aligned to upstream quota windows, plus manual Codex quota reset when a reset credit is available
• 🔌 Responses WebSocket - Downstream Codex WebSocket sessions bridge to native Codex WebSocket or HTTP/SSE candidates with transcript-aware failover
• 📦 Simple Deployment - Embedded SQLite; the Cursor SDK Bridge is managed automatically when needed
• 🔒 Secure Authentication - Token-based admin interface and API access control
• 🏷️ Build Tags - GOTAGS support, high-performance JSON library enabled by default
• 🐳 Docker Support - Multi-arch images (amd64/arm64), automated CI/CD
• ☁️ Cloud Native - Container deployment support, GitHub Actions auto-build
• 🤗 Hugging Face - One-click deployment to Hugging Face Spaces, free hosting
• 💰 Cost Limits - Per-channel daily cost limits, per-token cost limits
• 🚦 Channel RPM Limits - Per-channel rolling 60-second request caps, 0=unlimited
• 🚧 Channel Concurrency Limits - Per-channel in-flight request caps, 0=unlimited
• 🗝️ Per-Key Model Allowlists - Restrict which channel models each Key serves; empty means unrestricted, and channels whose Keys all decline the model are skipped
• 🧠 Model Thinking Suffix - Append (minimal/low/medium/high/xhigh/max), (none), (auto), or a numeric budget to any model name; ccLoad maps it to the upstream protocol's thinking parameters while routing on the base name
• 🖼️ Multimodal Fallback - Route requests containing images/files from non-vision models to configured fallback models (model_multimodal_fallback), applied before thinking-suffix handling and channel/Key selection
• 🕒 Channel Time Windows - Optional HH:MM availability window per channel (server local time, cross-midnight supported); channels outside their window are fully excluded from routing
• 🔐 Token Restrictions - Per-token cost limits, model restrictions, channel allowlist/denylist, and concurrency caps for fine-grained access control
• ⏱️ TTFB Monitoring - Streaming request first byte time tracking for upstream latency diagnosis
• 🌐 Multi-URL Load Balancing - Multiple URLs per channel with latency-weighted random selection
• 🧭 Per-Channel Proxy - Route a channel's upstream traffic through an http/https/socks5/socks5h proxy with isolated connection pools
• 💵 service_tier Pricing - OpenAI priority/flex/default tier multipliers for accurate cost accounting
• 🖼️ Image Tool Billing - Responses image_generation/gpt-image-2 cost accounting
• 📉 Tiered Pricing - GPT-5.4/Qwen-Plus/Gemini long-context step pricing, auto-applies lower rate at token thresholds
• 🔄 Per-URL Protocol Routing - Explicit Anthropic/OpenAI/Codex/Gemini capability per URL, with native-first automatic detection when left empty
• 💬 Conversational Model Testing - Channel/model/chat testing modes with image upload, reasoning level, built-in search, and chat export
• 🎨 Image Generation Testing - Dedicated tab that renders generated images through either the Images API or Chat Completions,