ccLoad

AI API gateway for Claude Code, Codex, Gemini, and OpenAI.

English | 简体中文

Smart routing | Automatic failover | Model-aware cooldown | Multi-URL scheduling | Protocol transforms | Live monitoring | Cost control

ccLoad removes the operational mess of running multiple AI API upstreams. It keeps Claude Code, Codex, Gemini, and OpenAI-compatible clients on one stable gateway, then handles upstream selection, failover, cooldown, protocol conversion, request visibility, and cost limits in the service instead of in every client script.

🤖 Built with Codex and GPT-5.6

During OpenAI Build Week, Codex powered by GPT-5.6 was the primary engineering agent used to:

• Trace request routing, failover, cooldown, protocol conversion, and dashboard flows across the Go backend and embedded web UI.
• Implement and review model-scoped cooldown handling for upstream 5xx, key-level 429, and model-unavailable 404 failures without unnecessarily cooling an entire channel.
• Refine the model-status and call-statistics UI, update the English and Chinese documentation, and verify the result with focused Go tests, builds, and browser walkthroughs.
• Prepare the reproducible demo and Devpost submission while keeping architecture, security, and final-review decisions under human control.

GPT-5.6 is also integrated into the product itself: ccLoad exposes GPT-5.6 through OpenAI-compatible and Codex Responses endpoints, includes Sol, Terra, and Luna model presets, calculates their standard, priority, flex, cached-token, and long-context costs, and applies routing and model-scoped cooldown decisions to them like any other configured upstream model.

The repository's AGENTS.md and CLAUDE.md provide persistent engineering constraints so Codex works against the same KISS-first review and testing rules in every session.

🎯 What ccLoad Solves

Common failure modes when you run several AI API channels:

• Manual channel switching: Different keys, validity windows, quotas, and upstream URLs quickly become hard to manage.
• Rate limits and upstream failures: 429, 502, 504, expired keys, and overloaded providers should not stop the client workflow.
• Opaque request status: Without live request visibility, long streaming requests become guesswork.
• HTTP 200 with error content: Some upstreams return a successful HTTP status while the response body is an actual error.
• Cost drift: Shared gateways need per-channel and per-token limits, not spreadsheet accounting after the bill arrives.

ccLoad handles those cases with:

• Smart routing: High-priority channels are selected first; channels at the same priority use smooth weighted round-robin.
• Automatic failover: Failed keys, models, channels, and URLs are skipped according to the classified error scope.
• Model-aware cooldown: Structured model_cooldown responses, upstream HTTP 5xx failures, key-level 429 rate limits, and model-unavailable 404 errors all cool only the actual upstream model first; other models on the same channel remain available. The channel is promoted to cooldown only after every configured model or every enabled key is cooling.
• Multi-URL scheduling: A single channel can use multiple upstream URLs, weighted by observed latency and health.
• Multi-protocol handling: Each channel has one primary protocol plus optional additional protocols, handled by upstream passthrough or ccLoad translation.
• Responses WebSocket bridging: Authenticated Codex clients can keep a downstream WebSocket while each candidate uses native Codex WebSocket or the existing HTTP/SSE transport.
• Live monitoring: Active requests, logs, token usage, TTFB, cost, and upstream details are visible in the web dashboard.
• Soft-error detection: HTTP 200 responses that are actually errors trigger the same failover path as regular upstream failures. Common cases include:
  • JSON responses containing {"error": {...}} structure
  • Responses with type field set to "error"
  • Explicit rate limits in SSE error events (rate_limit_exceeded / too_many_requests) are handled as 429
  • Plain text messages like "当前模型负载过高" / "Current model load too high" (load warnings)

✨ Key Features

• 🚀 High-Performance Architecture - Gin framework, 1000+ concurrent connections, high-performance caching
• 🧮 Local Token Counting - API-compliant local token estimation, <5ms response, 93%+ accuracy, supports large-scale tool scenarios
• 🎯 Smart Error Classification - Distinguishes Key/Model/Channel/Client errors, soft error detection (200 masquerading as error), SSE rate-limit errors as 429, 1308 quota handling
• 🔀 Smart Routing - Priority + smooth weighted round-robin channel selection, pre-filters cooled channels, multi-key load balancing, health-based dynamic sorting (confidence factor prevents small sample over-penalization)
• 🛡️ Failover - Key/channel failures use exponential backoff; model cooldowns (structured responses, 5xx, 429, model-unavailable 404) honor the upstream reset deadline and switch channels without cooling the whole channel
• 🔒 Race-Safe - Key selector race condition protection, startup config validation, automatic resource cleanup
• 📊 Real-time Monitoring - Built-in trend analysis, logging, and stats dashboard, Token usage stats with time range selection and per-token classification
• 🎯 Transparent Proxy - Supports Claude Code, Codex, Gemini, and OpenAI compatible APIs with smart auth detection
• 🔌 Responses WebSocket - Downstream Codex WebSocket sessions bridge to native Codex WebSocket or HTTP/SSE candidates with transcript-aware failover
• 📦 Single Binary Deployment - No external dependencies, embedded SQLite included
• 🔒 Secure Authentication - Token-based admin interface and API access control
• 🏷️ Build Tags - GOTAGS support, high-performance JSON library enabled by default
• 🐳 Docker Support - Multi-arch images (amd64/arm64), automated CI/CD
• ☁️ Cloud Native - Container deployment support, GitHub Actions auto-build
• 🤗 Hugging Face - One-click deployment to Hugging Face Spaces, free hosting
• 💰 Cost Limits - Per-channel daily cost limits, per-token cost limits
• 🚦 Channel RPM Limits - Per-channel rolling 60-second request caps, 0=unlimited
• 🚧 Channel Concurrency Limits - Per-channel in-flight request caps, 0=unlimited
• 🔐 Token Restrictions - Per-token cost limits, model restrictions, channel allowlist/denylist, and concurrency caps for fine-grained access control
• ⏱️ TTFB Monitoring - Streaming request first byte time tracking for upstream latency diagnosis
• 🌐 Multi-URL Load Balancing - Multiple URLs per channel with latency-weighted random selection
• 💵 service_tier Pricing - OpenAI priority/flex/default tier multipliers for accurate cost accounting
• 🖼️ Image Tool Billing - Responses image_generation/gpt-image-2 cost accounting
• 📉 Tiered Pricing - GPT-5.4/Qwen-Plus/Gemini long-context step pricing, auto-applies lower rate at token thresholds
• 🔄 Multi-Protocol Handling - One primary protocol plus additional protocols, with all 12 local conversion paths available when upstream passthrough is not appropriate
• 💬 Conversational Model Testing - Channel/model/chat testing modes with image upload, reasoning level, built-in search, and chat export
• 🔍 Debug Logs - Upstream request/response raw data capture with sensitive header masking, essential for troubleshooting
• 🕐 Scheduled Checks - Background periodic channel availability probing, auto-detect failed channels
• 🔄 Auto Updates - Checks for new releases every 12 hours by default, configurable from the admin settings page
• 🧩 Custom Request Rules - Per-channel HTTP header & JSON body rewriting (remove/override/append), with auth header protection, CRLF guard, and capacity caps
• 🎛️ Log Column Customization - Show/hide table columns per preference, settings persist in browser localStorage

🏗️ Architecture Overview

Each channel has one primary protocol and zero or more additional protocols. upstream (Upstream Passthrough) forwards each client protocol natively, while local (ccLoad Translation) converts additional protocols into the primary protocol at the Registry boundary.

graph TB
    subgraph "Client"
        A[Claude Code / Codex / Gemini / OpenAI Client]
    end

    subgraph "ccLoad Service"
        B[HTTP Proxy]
        C[Authentication + Route Dispatch]
        D[Channel Selector<br/>Priority + Smooth Weighted RR]
        E[Protocol Registry<br/>Native Bypass / Local Transform]
        F[URL Selector<br/>Explore + 1/EWMA Weighting]
        G[(Storage Factory<br/>SQLite / MySQL / PostgreSQL)]
        H[Logs + Metrics + Cost Control]

        A --> B --> C --> D --> E --> F
        D <--> G
        H <--> G
        B --> H
    end

    subgraph "Upstream Services"
        U1[Anthropic]
        U2[OpenAI-compatible]
        U3[Gemini]
        U4[Codex Responses]
    end

    F --> U1
    F --> U2
    F --> U3
    F --> U4
    U1 -. JSON / SSE .-> E
    U2 -. JSON / SSE .-> E
    U3 -. JSON / SSE .-> E
    U4 -. JSON / SSE .-> E
    E -. Client Protocol .-> B

    style B fill:#4F46E5,stroke:#000,color:#fff
    style D f