ThinkWatch solves all of this with a single deployment.

Key Features

AI API Gateway

• Multi-format API proxy — natively serves OpenAI Chat Completions (/v1/chat/completions), Anthropic Messages (/v1/messages), and OpenAI Responses (/v1/responses) APIs on a single port; works as a drop-in replacement for Cursor, Continue, Cline, Claude Code, and the OpenAI/Anthropic SDKs
• Multi-provider routing — OpenAI, Anthropic, Google Gemini, Azure OpenAI, AWS Bedrock, or any OpenAI-compatible endpoint
• Automatic format conversion — Anthropic Messages API, Google Gemini, Azure OpenAI, AWS Bedrock Converse API, and more, all behind a unified interface
• Provider auto-loading — active providers are loaded from the database at startup and registered in the model router; default model prefixes (gpt-/o1-/o3-/o4- for OpenAI, claude- for Anthropic, gemini- for Google) route automatically; Azure and Bedrock require explicit model registration
• Streaming SSE pass-through — zero-overhead forwarding with real-time token counting
• Virtual API keys — issue scoped tw- keys; the same tw- token works on both the AI gateway and the MCP gateway via a per-key surfaces allowlist
• API key lifecycle management — automatic rotation with grace periods, per-key inactivity timeout, expiry warnings, and background policy enforcement
• Composable rate limits & budgets — multi-window sliding limits (1m / 5m / 1h / 5h / 1d / 1w) and natural-period token budgets (daily / weekly / monthly), keyed per user, per API key, per provider, or per MCP server. See Rate limits & budgets below
• Per-model token weighting — gpt-4o tokens can count more than gpt-3.5 tokens against the same quota via configurable input_multiplier / output_multiplier
• Circuit breaker — three-state (Closed/Open/HalfOpen) circuit breaker with configurable failure threshold and recovery period
• Retry with exponential backoff — configurable retries with jitter for network errors and upstream rate limits
• Real-time cost tracking — per-model pricing with team attribution

MCP Gateway

• Centralized tool proxy — one MCP endpoint that aggregates tools from all upstream servers
• Namespace isolation — github__create_issue, postgres__query — no tool name collisions
• Tool-level RBAC — control exactly which users or roles can invoke which tools
• Connection pooling & health monitoring — automatic reconnection, background health checks
• Full audit trail — every tool invocation logged with user, parameters, and response

Security & Compliance

• Dual-port architecture — gateway (public-facing) and console (internal-only) on separate ports
• Role-based access control — 5-tier RBAC: Super Admin, Admin, Team Manager, Developer, Viewer
• SSO/OIDC — plug into Zitadel, Okta, Azure AD, or any OIDC-compliant provider
• AES-256-GCM encryption — provider API keys and secrets encrypted at rest
• SHA-256 key hashing — virtual API keys stored as hashes; plaintext shown exactly once
• Content Security Policy — CSP headers on the console port to prevent XSS and injection attacks
• JWT entropy enforcement — minimum 32-character secret with entropy validation at startup
• Startup dependency validation — verifies PostgreSQL, Redis, and encryption key availability with clear error messages before accepting traffic
• Security headers — X-Content-Type-Options, X-Frame-Options, CORS whitelisting, request timeouts
• Soft-delete — users, providers, and API keys use soft-delete (deleted_at column) with automatic purge after 30 days
• Password complexity — minimum 8 characters with required uppercase, lowercase, and digit
• Session IP binding — admin sessions bound to client IP; stolen tokens cannot be replayed from a different network
• Distroless containers — minimal attack surface in production (2MB runtime image, no shell)

Operations & Configuration

• Dynamic configuration — most settings stored in database (system_settings table), configurable via Web UI (Admin > Settings with 7 category tabs)
• First-run setup wizard — guided /setup wizard creates the super_admin account, configures the site, and optionally adds the first provider and API key
• Configuration Guide — built-in /gateway/guide page in the web console with copy-paste setup instructions for Claude Code, Cursor, Continue, Cline, OpenAI SDK, Anthropic SDK, and cURL; auto-detects the gateway URL
• Multi-instance sync — configuration changes propagated across instances via Redis Pub/Sub
• Data retention policies — configurable retention periods for usage records and audit logs with automatic daily purge

Observability

• Prometheus metrics — GET /metrics endpoint on the gateway port (3000) exposing gateway_requests_total, gateway_request_duration_seconds, gateway_tokens_total, gateway_rate_limited_total, circuit_breaker_state, and more
• Enhanced health checks — /health/live (liveness probe), /health/ready (readiness probe with PostgreSQL and Redis checks), /api/health (detailed latency and pool statistics)
• ClickHouse-powered audit logs — SQL-queryable audit logs across all API calls and tool invocations, stored in ClickHouse for high-performance columnar analytics
• Audit log forwarding — multi-channel delivery: UDP/TCP Syslog (RFC 5424), Kafka, and HTTP webhooks — route audit events to any SIEM, data lake, or alerting pipeline
• Usage analytics — token consumption by user, team, model, and time period
• Cost analytics — MTD spend, budget utilization, per-model cost breakdown
• Health dashboard — real-time status of PostgreSQL, Redis, ClickHouse, and all MCP servers
• Unified log explorer — search across audit, gateway, MCP, access, and platform logs from a single page with structured query syntax

Rate limits & budgets

ThinkWatch enforces two parallel kinds of quota at every gateway request, both managed from the same admin UI:

| | Sliding-window rate limits | Natural-period budget caps | |---|---|---| | What it counts | Requests OR weighted tokens, depending on the rule's metric | Weighted tokens only | | Window shape | Rolling 60-bucket window: 1m / 5m / 1h / 5h / 1d / 1w | Calendar-aligned: daily / weekly / monthly (resets on the period boundary) | | Backing store | Redis ZSET-style buckets | Redis INCR counters keyed by subject:period:bucket_id | | When it fires | Pre-flight (requests metric) AND post-flight (tokens metric) | Post-flight only | | Hard or soft? | Hard for requests metric, soft for tokens metric | Soft cap — exactly one request can push you over before subsequent calls in the same period are rejected |

Subjects

A single request can be subject to multiple rules and budgets at once. The engine resolves the request to a set of (subject_kind, subject_id) tuples and runs every enabled rule against all of them in one atomic Lua check. Any rule rejecting → the request is rejected. All-or-nothing INCR.

| Subject | Rate limit rules | Budget caps | |---|---|---| | user | ✅ ai_gateway / mcp_gateway | ✅ | | api_key | ✅ ai_gateway / mcp_gateway | ✅ | | provider | ✅ ai_gateway only | ✅ | | mcp_server | ✅ mcp_ga