ccLoad

AI API gateway for Claude Code, Codex, Gemini, and OpenAI.

English | 简体中文

Smart routing | Automatic failover | Exponential cooldown | Multi-URL scheduling | Protocol transforms | Live monitoring | Cost control

ccLoad removes the operational mess of running multiple AI API upstreams. It keeps Claude Code, Codex, Gemini, and OpenAI-compatible clients on one stable gateway, then handles upstream selection, failover, cooldown, protocol conversion, request visibility, and cost limits in the service instead of in every client script.

🎯 What ccLoad Solves

Common failure modes when you run several AI API channels:

• Manual channel switching: Different keys, validity windows, quotas, and upstream URLs quickly become hard to manage.
• Rate limits and upstream failures: 429, 502, 504, expired keys, and overloaded providers should not stop the client workflow.
• Opaque request status: Without live request visibility, long streaming requests become guesswork.
• HTTP 200 with error content: Some upstreams return a successful HTTP status while the response body is an actual error.
• Cost drift: Shared gateways need per-channel and per-token limits, not spreadsheet accounting after the bill arrives.

ccLoad handles those cases with:

• Smart routing: High-priority channels are selected first; channels at the same priority use smooth weighted round-robin.
• Automatic failover: Failed keys, channels, and URLs are skipped according to the classified error type.
• Exponential cooldown: Unhealthy upstreams back off automatically instead of being hammered by retries.
• Multi-URL scheduling: A single channel can use multiple upstream URLs, weighted by observed latency and health.
• Protocol transforms: Anthropic, OpenAI, Gemini, and Codex request/response families can be converted at the gateway.
• Live monitoring: Active requests, logs, token usage, TTFB, cost, and upstream details are visible in the web dashboard.
• Soft-error detection: HTTP 200 responses that are actually errors trigger the same failover path as regular upstream failures. Common cases include:
  • JSON responses containing {"error": {...}} structure
  • Responses with type field set to "error"
  • Explicit rate limits in SSE error events (rate_limit_exceeded / too_many_requests) are handled as 429
  • Plain text messages like "当前模型负载过高" / "Current model load too high" (load warnings)

✨ Key Features

• 🚀 High-Performance Architecture - Gin framework, 1000+ concurrent connections, high-performance caching
• 🧮 Local Token Counting - API-compliant local token estimation, <5ms response, 93%+ accuracy, supports large-scale tool scenarios
• 🎯 Smart Error Classification - Distinguishes Key/Channel/Client errors, soft error detection (200 masquerading as error), SSE rate-limit errors as 429, 1308 quota handling
• 🔀 Smart Routing - Priority + smooth weighted round-robin channel selection, pre-filters cooled channels, multi-key load balancing, health-based dynamic sorting (confidence factor prevents small sample over-penalization)
• 🛡️ Failover - Automatic failure detection with exponential backoff cooldown (1s → 2s → 4s → ... → 30min)
• 🔒 Race-Safe - Key selector race condition protection, startup config validation, automatic resource cleanup
• 📊 Real-time Monitoring - Built-in trend analysis, logging, and stats dashboard, Token usage stats with time range selection and per-token classification
• 🎯 Transparent Proxy - Supports Claude Code, Codex, Gemini, and OpenAI compatible APIs with smart auth detection
• 📦 Single Binary Deployment - No external dependencies, embedded SQLite included
• 🔒 Secure Authentication - Token-based admin interface and API access control
• 🏷️ Build Tags - GOTAGS support, high-performance JSON library enabled by default
• 🐳 Docker Support - Multi-arch images (amd64/arm64), automated CI/CD
• ☁️ Cloud Native - Container deployment support, GitHub Actions auto-build
• 🤗 Hugging Face - One-click deployment to Hugging Face Spaces, free hosting
• 💰 Cost Limits - Per-channel daily cost limits, per-token cost limits
• 🚦 Channel RPM Limits - Per-channel rolling 60-second request caps, 0=unlimited
• 🚧 Channel Concurrency Limits - Per-channel in-flight request caps, 0=unlimited
• 🔐 Token Restrictions - API token cost limits + model restrictions for fine-grained access control
• ⏱️ TTFB Monitoring - Streaming request first byte time tracking for upstream latency diagnosis
• 🌐 Multi-URL Load Balancing - Multiple URLs per channel with latency-weighted random selection
• 💵 service_tier Pricing - OpenAI priority/flex/default tier multipliers for accurate cost accounting
• 🖼️ Image Tool Billing - Responses image_generation/gpt-image-2 cost accounting
• 📉 Tiered Pricing - GPT-5.4/Qwen-Plus/Gemini long-context step pricing, auto-applies lower rate at token thresholds
• 🔄 Protocol Transform - Anthropic/OpenAI/Gemini/Codex cross-protocol conversion, preserving sampling and thinking parameters so one channel serves multiple client protocols
• 🔍 Debug Logs - Upstream request/response raw data capture with sensitive header masking, essential for troubleshooting
• 🕐 Scheduled Checks - Background periodic channel availability probing, auto-detect failed channels
• 🧩 Custom Request Rules - Per-channel HTTP header & JSON body rewriting (remove/override/append), with auth header protection, CRLF guard, and capacity caps
• 🎛️ Log Column Customization - Show/hide table columns per preference, settings persist in browser localStorage

🏗️ Architecture Overview

graph TB
    subgraph "Client"
        A[User App] --> B[ccLoad Proxy]
    end

    subgraph "ccLoad Service"
        B --> C[Auth Layer]
        C --> D[Route Dispatcher]
        D --> E[Channel Selector]
        E --> F[Load Balancer]

        subgraph "Core Components"
            F --> G[Channel A<br/>Priority:10]
            F --> H[Channel B<br/>Priority:5]
            F --> I[Channel C<br/>Priority:5]
            G --> G1[URL Selector<br/>Weighted Random]
            H --> H1[URL Selector<br/>Weighted Random]
            I --> I1[URL Selector<br/>Weighted Random]
        end

        subgraph "Storage Layer"
            J[(Storage Factory)]
            J3[Schema Definition]
            J4[Unified SQL Layer]
            J1[(SQLite)]
            J2[(MySQL)]
            J --> J3
            J3 --> J4
            J4 --> J1
            J4 --> J2
        end

        subgraph "Monitoring Layer"
            K[Log System]
            L[Stats Analysis]
            M[Trend Charts]
        end
    end

    subgraph "Upstream Services"
        G1 --> N[Claude API]
        H1 --> O[Claude API]
        I1 --> P[Claude API]
    end

    E <--> J
    F <--> J
    K <--> J
    L <--> J
    M <--> J

    style B fill:#4F46E5,stroke:#000,color:#fff
    style F fill:#059669,stroke:#000,color:#fff
    style E fill:#0EA5E9,stroke:#000,color:#fff

🚀 Quick Start

Choose the deployment method that suits you best:

Method 1: Docker Deployment (Recommended)

Using pre-built images (Recommended):

# Option 1: Using docker-compose (Simplest)
curl -o docker-compose.yml https://raw.githubusercontent.com/caidaoli/ccLoad/master/docker-compose.yml
curl -o .env https://raw.githubusercontent.com/caidaoli/ccLoad/master/.env.example
# Edit .env file to set password
docker-compose up -d

# Option 2: Run image directly
docker pull ghcr.io/caidaoli/ccload:latest
docker run -d --name ccload \
  -p 8080:8080 \
  -e CCLOAD_PASS=your_secure_password \
  -v ccload_data:/app/data \
  ghcr.io/caidaoli/ccload:latest

Building from source:

# Clone project
git clone https://github.com/caidaoli/ccLoad.git
cd ccLoad

# Build and run with docker-compose
docker-compose -f docker-compose.build.yml up -d

# Or build manually
docker build -t ccload:local .
docker run -d --name ccload \
  -p 8080:8080 \
  -e CCLOAD_PASS=your_secure_password \
  -v ccload_data:/app/data \
  ccload:local

Method 2: Source Build

# Clone project
git clone https://github.com/caidaoli/ccLoad.git
cd ccLoad

# Build project (uses high-performance JSON library by default)
go build -tags sonic -o ccload .

# Or use Makefile
make build

# Run in development mode
go run -tags sonic .
# Or
make dev

Method 3: Binary Download

# Download binary for your platform from GitHub Releases
wget https://github.com/caidaoli/ccLoad/releases/latest/download/ccload-linux-amd64
chmod +x ccload-linux-amd64
./ccload-linux-amd64

Method 4: Hugging Face Spaces Deployment

Hugging Face Spaces provides free container hosting with Docker support, ideal for per